Import : College_Admissions_IPEDS_National : auto refresh config #1819

vishalworkdatacommon · 2025-12-30T13:56:36Z

No description provided.

gemini-code-assist · 2025-12-30T13:57:20Z

Summary of Changes

Hello @vishalworkdatacommon, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request establishes a new, fully automated data import for national college admissions and enrollment figures from the IPEDS database. The changes introduce a robust pipeline capable of downloading raw data, processing it through a statistical variable processor, and generating ready-to-ingest artifacts. This ensures up-to-date and consistent data availability for a wide range of academic years.

Highlights

New Data Import: Introduced a new automated import for national-level college admissions and enrollment data from the Integrated Postsecondary Education Data System (IPEDS).
Comprehensive Data Coverage: The import covers academic years from 2014-15 up to 2023-24, providing a decade of historical data.
Automated Pipeline: Implemented an automated pipeline using download_script.py for data acquisition and stat_var_processor.py for preprocessing and artifact generation, configured for regular refreshes.
Detailed Documentation: A new README.md file provides a comprehensive overview of the import, including preprocessing steps, autorefresh details, and instructions for manual execution.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request adds a new automated import for College Admissions data from IPEDS. The changes are well-structured, including documentation, configuration files, processing scripts, and test data.

My review focuses on improving maintainability and consistency. I've identified several areas with code/config duplication in manifest.json and the pv_map files, which could be consolidated. There are also some inconsistencies between the manual run.sh script and the automated configuration in manifest.json that should be addressed. Finally, I've pointed out a few typos in the README.md file.

statvar_imports/ipeds/college_admission_national/run.sh

statvar_imports/ipeds/college_admission_national/README.md

gemini-code-assist · 2025-12-30T14:00:00Z

statvar_imports/ipeds/college_admission_national/manifest.json

+                "../../tools/statvar_importer/stat_var_processor.py --existing_statvar_mcf=admissions_stat_vars_common.mcf --input_data=input_files/college_admissions_2014.csv --pv_map=pv_map/college_admissions_ipeds_pv_map_2014.csv --config_file=college_admissions_ipeds_metadata.csv --output_path=output_files/admissions_output_2014",
+                "../../tools/statvar_importer/stat_var_processor.py --existing_statvar_mcf=admissions_stat_vars_common.mcf --input_data=input_files/college_admissions_2015.csv --pv_map=pv_map/college_admissions_ipeds_pv_map_2015.csv --config_file=college_admissions_ipeds_metadata.csv --output_path=output_files/admissions_output_2015",
+                "../../tools/statvar_importer/stat_var_processor.py --existing_statvar_mcf=admissions_stat_vars_common.mcf --input_data=input_files/college_admissions_2016.csv --pv_map=pv_map/college_admissions_ipeds_pv_map_2016.csv --config_file=college_admissions_ipeds_metadata.csv --output_path=output_files/admissions_output_2016",
+                "../../tools/statvar_importer/stat_var_processor.py --existing_statvar_mcf=admissions_stat_vars_common.mcf --input_data=input_files/college_admissions_2017.csv --pv_map=pv_map/college_admissions_ipeds_pv_map_2017.csv --config_file=college_admissions_ipeds_metadata.csv --output_path=output_files/admissions_output_2017",
+                "../../tools/statvar_importer/stat_var_processor.py --existing_statvar_mcf=admissions_stat_vars_common.mcf --input_data=input_files/college_admissions_2018.csv --pv_map=pv_map/college_admissions_ipeds_pv_map_2018.csv --config_file=college_admissions_ipeds_metadata.csv --output_path=output_files/admissions_output_2018",
+                "../../tools/statvar_importer/stat_var_processor.py --existing_statvar_mcf=admissions_stat_vars_common.mcf --input_data=input_files/college_admissions_2019.csv --pv_map=pv_map/college_admissions_ipeds_pv_map_2019.csv --config_file=college_admissions_ipeds_metadata.csv --output_path=output_files/admissions_output_2019",
+                "../../tools/statvar_importer/stat_var_processor.py --existing_statvar_mcf=admissions_stat_vars_common.mcf --input_data=input_files/college_admissions_2020.csv --pv_map=pv_map/college_admissions_ipeds_pv_map_2020.csv --config_file=college_admissions_ipeds_metadata.csv --output_path=output_files/admissions_output_2020",
+                "../../tools/statvar_importer/stat_var_processor.py --existing_statvar_mcf=admissions_stat_vars_common.mcf --input_data=input_files/college_admissions_2021.csv --pv_map=pv_map/college_admissions_ipeds_pv_map_2021.csv --config_file=college_admissions_ipeds_metadata.csv --output_path=output_files/admissions_output_2021",
+                "../../tools/statvar_importer/stat_var_processor.py --existing_statvar_mcf=admissions_stat_vars_common.mcf --input_data=input_files/college_admissions_2022.csv --pv_map=pv_map/college_admissions_ipeds_pv_map_2022.csv --config_file=college_admissions_ipeds_metadata.csv --output_path=output_files/admissions_output_2022",
+                "../../tools/statvar_importer/stat_var_processor.py --existing_statvar_mcf=admissions_stat_vars_common.mcf --input_data=input_files/college_admissions_2023.csv --pv_map=pv_map/college_admissions_ipeds_pv_map_2023.csv --config_file=college_admissions_ipeds_metadata.csv --output_path=output_files/admissions_output_2023"


The scripts array contains many repetitive calls to stat_var_processor.py. This violates the DRY (Don't Repeat Yourself) principle and makes the manifest harder to maintain when adding new years. Consider creating a Python script that loops through the years (similar to run.sh) and calling that single script from here. This would simplify the manifest significantly.

statvar_imports/ipeds/college_admission_national/manifest.json

...var_imports/ipeds/college_admission_national/pv_map/college_admissions_ipeds_pv_map_2014.csv

statvar_imports/ipeds/college_admission_national/run.sh

statvar_imports/ipeds/college_admission_national/download_script.py

krishnaswamypradeep · 2025-12-31T06:27:39Z

statvar_imports/ipeds/college_admission_national/manifest.json

+            "provenance_description": "Data on student applications, admissions, and enrollment for postsecondary institutions across the United States.",
+            "scripts": [
+                "download_script.py",
+                "../../tools/statvar_importer/stat_var_processor.py --existing_statvar_mcf=admissions_stat_vars_common.mcf --input_data=input_files/college_admissions_2014.csv --pv_map=pv_map/college_admissions_ipeds_pv_map_2014.csv --config_file=college_admissions_ipeds_metadata.csv --output_path=output_files/admissions_output_2014",


admissions_stat_vars_common.mcf file is missing

statvar_imports/ipeds/college_admission_national/run.sh

auto refresh config

bd40e1b

adding download_script.py

6f19bec

gemini-code-assist bot reviewed Dec 30, 2025

View reviewed changes

krishnaswamypradeep reviewed Dec 31, 2025

View reviewed changes

vishalworkdatacommon added 2 commits December 31, 2025 08:03

addressing pr comments

9ae9544

addressing unused file

4896984

krishnaswamypradeep approved these changes Dec 31, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Import : College_Admissions_IPEDS_National : auto refresh config #1819

Import : College_Admissions_IPEDS_National : auto refresh config #1819

Uh oh!

vishalworkdatacommon commented Dec 30, 2025

Uh oh!

gemini-code-assist bot commented Dec 30, 2025

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

Uh oh!

gemini-code-assist bot Dec 30, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

krishnaswamypradeep Dec 31, 2025

Uh oh!

vishalworkdatacommon Dec 31, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Import : College_Admissions_IPEDS_National : auto refresh config #1819

Are you sure you want to change the base?

Import : College_Admissions_IPEDS_National : auto refresh config #1819

Uh oh!

Conversation

vishalworkdatacommon commented Dec 30, 2025

Uh oh!

gemini-code-assist bot commented Dec 30, 2025

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

gemini-code-assist bot Dec 30, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

krishnaswamypradeep Dec 31, 2025

Choose a reason for hiding this comment

Uh oh!

vishalworkdatacommon Dec 31, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants