fars-cleaner is a Python library for downloading and pre-processing data
from the Fatality Analysis Reporting System, collected annually by NHTSA since
1975.
The preferred installation method is through conda.
conda install -c conda-forge fars_cleanerYou can also install with pip.
pip install fars-cleanerThe FARSFetcher class provides an interface to download and unzip selected years from the NHTSA FARS FTP server.
The downloads include progress bars for both downloading and extracting files. By default, files are stored in a .cache/fars directory within your home directory.
from fars_cleaner import FARSFetcher
# Prepare for FARS file download, using the default cache directory.
# Progress bars will be displayed during download and extraction
fetcher = FARSFetcher(show_progress=True)Suggested usage is to download files to a data directory in your current project directory.
Passing project_dir will download files to project_dir/data/fars by default. This behavior can be
overridden by setting cache_path as well. Setting cache_path alone provides a direct path to the directory
you want to download files into.
from pathlib import Path
from fars_cleaner import FARSFetcher
SOME_PATH = Path("/YOUR/PROJECT/PATH")
# Prepare to download to /YOUR/PROJECT/PATH/data/fars
# This is the recommended usage.
fetcher = FARSFetcher(project_dir=SOME_PATH)
# Prepare to download to /YOUR/PROJECT/PATH/fars
cache_path = "fars"
fetcher = FARSFetcher(project_dir=SOME_PATH, cache_path=cache_path)
cache_path = Path("/SOME/TARGET/DIRECTORY")
# Prepare to download directly to a specific directory.
fetcher = FARSFetcher(cache_path=cache_path)Files can be downloaded in their entirety (data from 1975-2018), as a single year, or across a specified year range.
Downloading all of the data can be quite time consuming. The download will simultaneously unzip the folders, and delete
the zip files. Each zipped file will be unzipped and saved in a folder {YEAR}.unzip
# Fetch all data
fetcher.fetch_all()
# Fetch a single year
fetcher.fetch_single(1984)
# Fetch data in a year range (inclusive).
fetcher.fetch_subset(1999, 2007)Calling load_pipeline will allow for full loading and pre-processing of the FARS data requested by the user.
from fars_cleaner import FARSFetcher, load_pipeline
fetcher = FARSFetcher(project_dir=SOME_PATH)
vehicles, accidents, people = load_pipeline(fetcher=fetcher,
first_run=True,
target_folder=SOME_PATH)Calling load_basic allows for simple loading of the FARS data for a single year, with no preprocessing. Files must
be prefetched using a FARSFetcher or similar method. A mapper dictionary must be provided to identify what, if
any, columns require renaming.
from fars_cleaner.data_loader import load_basic
vehicles, accidents, people = load_basic(year=1975, data_dir=SOME_PATH, mapping=mappings)Downloading and processing the full FARS dataset currently runs out of memory on Windows machines with only 16GB RAM. It is recommended to have at least 32GB RAM on Windows systems. macOS and Linux run with no issues on 16GB systems.
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change. See CONTRIBUTING.md for more details.
- 1.4.0 - Removed dependency on Pooch library. Now using direct downloads with progress bars via requests and tqdm.
- 1.3.5 - Previous version