Analyzer

Analyzer

What is the Analyzer?

The Analyzer is a tool that checks if the profiles collected by the Crawler are likely to be NAF alumni.

What has been done so far?

• Collected data to train a machine learning (ML) model (data is available here: https://docs.google.com/spreadsheets/d/1BBxZNuxyXkUHUc9krwzmN3ktNMM6yVH4NNTZzVfhs9w/edit?usp=sharing)
• Built an ML model that assigns importance (weights) to different profile identifiers
• Created a way to calculate a confidence score based on these weights
• Connected the Analyzer to the database so it can read profiles from the Crawler and save results in the Analyzer tables

How to run the Analyzer

The Analyzer takes profiles from the Crawler table, analyzes them, and stores results in the unconfirmed_alumni table.

1. Open your terminal and go to the Analyzer folder:

 cd analyzer

2. Build the TypeScript code into JavaScript:

  npm run build

3. Start the Analyzer:

  npm start

What to expect

• The program will skip profiles already analyzed (based on the profile_url)
• It will show how many new profiles were analyzed

Example output:

What’s next? (Spring 2025 goals)

• Collect more training data to improve the ML model, especially for rare cases
• Fine-tune confidence score thresholds to improve accuracy

Where to find more documentation

All related documents are stored here: https://drive.google.com/drive/folders/1pZor1ZZ0oMw0NGoDMwU8nDXF-GsEUvMC?usp=drive_link