Availability of precomputed AlphaFold3 features (JSON format) + modelcif crashes for precomputed features (bug)

[abiadak]

Hi,

thanks for providing precomputed AlphaFold2 feature for model organisms, they are very useful.

I’m currently looking into using AlphaPulldown with AlphaFold3 backend, which requires JSON-formatted input features. I’ve searched through the existing issues in this repository but couldn’t find a clear answer to this point.

Is there any public repository or resource where precomputed AlphaFold3 features (JSON) are available?
If not, do you plan to provide such features in the future, or should users assume that features must always be generated locally with the AF3 pipeline?

Thanks in advance for the clarification.

[DimaMolod]

Hi @abiadak, thanks for your question.

At the moment, we do not provide precomputed AlphaFold3 (AF3) features in JSON format. I’ll let @jkosinski comment on whether this is something we may support in the future.

That said, the AlphaFold3 backend in AlphaPulldown is compatible with AlphaFold2-style features. In practice, you can also use the standard AF2 .pkl (or zipped pkl.xz) feature files as input.

While the AF2 databases are slightly older, I do not expect major differences in prediction accuracy when using AF2- and AF3-generated features. But we haven't tested it systematically, only on a couple of examples.

Happy to help,
Dima

[jkosinski]

At the moment, we do not provide precomputed AlphaFold3 (AF3) features in JSON format. I’ll let @jkosinski comment on whether this is something we may support in the future.

Yes, I think at some point we might provide the precomputed AF3 features, so let's keep it open as a feature request.

[abiadak]

Thanks a lot for the clarification — that’s very helpful.

I wasn’t aware that the AlphaFold3 backend in AlphaPulldown is compatible with AlphaFold2-style features. That definitely simplifies things.

I’ve run into a related practical issue when using the precomputed features. In the feature_metadata.json.xz files, the FASTA paths are hard-coded, e.g.:

/scratch/dima/features_DBs/Escherichia_coli/data/U3PVA8.fasta

When running AlphaPulldown via run_multimer_jobs.py, after the prediction step it tries to access the FASTA file at that exact location. I couldn’t find any flag or option to override the FASTA base path, so the workaround I used was to edit those paths in the metadata files to point to my local FASTA directory.

Is there a supported way to provide or override the FASTA path at runtime, without modifying the metadata.json.xz files themselves?

Thanks again for your help.

[DimaMolod]

Hi @abiadak
the metadata JSON files are not used to define paths, they are there just for your information. If meta file is not found, the pipeline throws a warning message, which is not critical and does not stop the execution.
Seems like you define the wrong path to your features (--features_directory):
/scratch/dima/ <- this can't be right.

[abiadak]

Hi Dima,

This is how the job is running, with the Alphafold2 backend and with the provided features:

I0204 19:39:59.410406 140481269221184 run_multimer_jobs.py:179] command: ['/lustre/home/cbm/opt/AlphaPulldown/bin/python3', '/lustre/home/cbm/opt/AlphaPulldown/bin/run_structure_prediction.py', '--output_directory', 'ETFDH_com
p', '--num_cycle', '3', '--num_predictions_per_model', '1', '--data_directory', '/lustre/databases/alphafold_db', '--features_directory', '/lustre/home/cbm/databases/Mus_musculus_tmp/', '--pair_msa', '', '--nomsa_depth_scan', '', '-
-nomultimeric_template', '', '--fold_backend', 'alphafold', '--nocompress_result_pickles', '', '--noremove_result_pickles', '', '--remove_keys_from_pickles', '', '--use_ap_style', '', '--use_gpu_relax', '', '--protein_delimiter', '+', '--
models_to_relax', 'None', '--input', 'A0A075B685+A0A075B685']

This is the error that appears at the log:

I0204 20:02:15.555695 139977075447616 alphafold_backend.py:718] Converting ETFDH_comp/A0A075B685_homo_2er to ModelCIF format...
E0204 20:02:20.242147 139977075447616 alphafold_backend.py:729] Error: E0204 20:02:20.152105 140057253898048 convert_to_modelcif.py:1410] Error while processing model 'ETFDH_comp/A0A075B685_homo_2er/ranked_3.pdb' of complex 'A0A075B685_homo_2er': [Errno 2] No such file or directory: '/scratch/dima/features_DBs/Mus_musculus/data/A0A075B685.fasta'

And the .cif files are not created.
This is the content of the metadata file, where appears the path "/scratch/dima/features_DBs/Mus_musculus/data/A0A075B685.fasta"

xzcat /lustre/home/cbm/databases/Mus_musculus_tmp/A0A075B685_feature_metadata_2024-09-17.json.xz 
{"databases": {"UniProt": {"release_date": "2024-08-28 10:19:37", "version": null, "location_url": ["ftp://ftp.ebi.ac.uk/pub/databases/uniprot/current_release/knowledgebase/complete/uniprot_trembl.fasta.gz", "ftp://ftp.ebi.ac.uk/pub/databases/uniprot/current_release/knowledgebase/complete/uniprot_sprot.fasta.gz"]}, "PDB seqres": {"release_date": "2024-09-01 04:39:00", "version": "546c661e9ffdb6f035040d28176f1c5a", "location_url": ["ftp://ftp.wwpdb.org/pub/pdb/derived_data/pdb_seqres.txt"]}, "ColabFold": {"version": "2024-09-17", "release_date": null, "location_url": ["https://wwwuser.gwdg.de/~compbiol/colabfold/colabfold_envdb_202108.tar.gz"]}}, "software": {"AlphaPulldown": {"version": "2.0.0.b5"}, "AlphaFold": {"version": "2.3.2"}, "jackhmmer": {"version": "3.4"}, "hhblits": {"version": "3.3.0"}, "hhsearch": {"version": "3.3.0"}, "hmmsearch": {"version": "3.4"}, "hmmbuild": {"version": "3.4"}, "kalign": {"version": "2.04"}}, "date": "2024-09-17 16:12:26", "other": {"logtostderr": "False", "alsologtostderr": "False", "log_dir": "", "v": "0", "verbosity": "0", "logger_levels": "{}", "stderrthreshold": "fatal", "showprefixforinfo": "True", "run_with_pdb": "False", "pdb_post_mortem": "False", "pdb": "False", "run_with_profiling": "False", "only_check_args": "False", "xml_output_file": "", "op_conversion_fallback_to_while_loop": "True", "delta_threshold": "0.5", "tt_check_filter": "False", "tt_single_core_summaries": "False", "runtime_oom_exit": "True", "hbm_oom_exit": "True", "fasta_paths": "['/scratch/dima/features_DBs/Mus_musculus/data/A0A075B685.fasta']", "jackhmmer_binary_path": "/opt/conda/bin/jackhmmer", "hhblits_binary_path": "/usr/bin/hhblits", "hhsearch_binary_path": "/usr/bin/hhsearch", "hmmsearch_binary_path": "/opt/conda/bin/hmmsearch", "hmmbuild_binary_path": "/opt/conda/bin/hmmbuild", "kalign_binary_path": "/opt/conda/bin/kalign", "uniprot_database_path": "/scratch_cached/AlphaFold_DBs/2.3.2/uniprot/uniprot.fasta", "pdb_seqres_database_path": "/scratch_cached/AlphaFold_DBs/2.3.2/pdb_seqres/pdb_seqres.txt", "template_mmcif_dir": "/scratch_cached/AlphaFold_DBs/2.3.2/pdb_mmcif/mmcif_files", "obsolete_pdbs_path": "/scratch_cached/AlphaFold_DBs/2.3.2/pdb_mmcif/obsolete.dat", "db_preset": "full_dbs", "model_preset": "monomer", "benchmark": "False", "num_multimer_predictions_per_model": "5", "use_precomputed_msas": "False", "models_to_relax": "ModelsToRelax.BEST", "use_mmseqs2": "False", "save_msa_files": "False", "skip_existing": "False", "use_hhsearch": "False", "compress_features": "False", "threshold_clashes": "1000.0", "hb_allowance": "0.4", "plddt_threshold": "0.0", "multiple_mmts": "False", "use_small_bfd": "False"}}

When I fix the *_feature_metadata_2024-09-17.json.xz files with the paths to where the fasta files are stored in my file system, the error disappear and the .cif files are created.

[DimaMolod]

Ok, that seems to be a problem of using pre-computed features from our features database.
Yes, the modelcif format requires query sequence and of course you don't have the original fasta files locally.
No, unfortunately there are no AP flags to override it. The predictions as pdb files should still be there, though.
One temporal way around it is indeed to modify the metadata files manually or to create symlinks.
I'll label this as a bug, thank you @abiadak for pointing this out.

[abiadak]

Thanks a lot for considering this and for labeling it. I appreciate you taking note for a future fix!