Optimize DPL classifier with pre-serialized snapshot#113
Open
lucasfeliciano wants to merge 11 commits intoPauperformance:masterfrom
Open
Optimize DPL classifier with pre-serialized snapshot#113lucasfeliciano wants to merge 11 commits intoPauperformance:masterfrom
lucasfeliciano wants to merge 11 commits intoPauperformance:masterfrom
Conversation
Pre-serialize classifier state to a single pickle file to reduce cold start time from minutes to under 5 seconds, enabling potential Lambda/serverless deployment.
8 tasks covering: Decklassifier changes, build script, snapshot loading, silver.py updates, Makefile target, and testing.
…om Scryfall Add _artifact_land_names field to Decklassifier and a from_snapshot_data classmethod that constructs a Decklassifier without requiring PauperformanceService. The _is_affinity method now uses pre-loaded artifact land names when available, falling back to Scryfall otherwise. Move PauperformanceService and MTGGoldfish imports behind TYPE_CHECKING and local scope respectively to allow importing Decklassifier without triggering heavy service initialization side effects.
…nceService Also fix _cosine_similarity to handle empty vectors (e.g. empty sideboards) by returning 1 when vector length is 0, preventing NaN from scipy.
…s None When classify_deck encounters an uncached reference deck and pauperformance is None (snapshot mode), skip it instead of crashing. This allows the classifier to fall back to known_decks for matching.
Move heavy imports inside get_dpl_classifier() so they're only loaded on fallback. Add load_dpl_classifier() that tries snapshot first.
- Extract CLASSIFIER_SNAPSHOT_PATH to shared constant in silver.py - Add os.makedirs in build_snapshot for robustness - Add negative Affinity detection test - Add .gitignore entry for classifier_snapshot.pkl - Fix duplicate CLASSIFIER_ prefix from replace_all
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
from_snapshot_dataandfrom_snapshotclassmethods toDecklassifierfor loading from pre-built pickle filesbuild_snapshot.pyscript that runs the expensive initialization once and serializes all classifier state (archetypes, training decks, reference deck cache, artifact land names) into a single filesilver.pyto load from snapshot on startup with fallback to full initialization_is_affinityfrom Scryfall API by supporting pre-loaded artifact land namesmake build-snapshottargetThis reduces classifier cold start from minutes to under 5 seconds by eliminating runtime dependencies on Dropbox, Scryfall, and MTGGoldfish. Enables potential Lambda/serverless deployment.
Test Plan
test_decklassifier.pycovering snapshot creation, loading, classification, and edge casesmake build-snapshoton a machine with Dropbox/MTGGoldfish credentials to generate the snapshot