`mlflow run {any url to this repo}` fails with "fatal: repository not found"

[hdotking]

Details

Code (cloned the repo as instructed): https://github.com/PacktPublishing/Machine-Learning-Engineering-with-MLflow/tree/master/Chapter01/stockpred
OS Platform and Distribution: Windows 10 - 64 bit
MLflow installed from (source or binary): conda create --channel conda-forge --name mlflow_env mlflow
MLflow version (run mlflow --version): 1.20.2
Python version: 3.7.12
Exact command to reproduce: mlflow run https://github.com/PacktPublishing/Machine-Learning-Engineering-with-MLflow/tree/master/Chapter01/stockpred
Describe the problem: After installing mlflow with conda-forge I am unable to run any of the examples in this repository. External repos work fine (See bottom of page)
Expected behaviour: Python file is executed and tracked and run is added in mlruns.
Actual behaviour: mlflow/GIT throws an error -->

2021/11/04 14:11:24 INFO mlflow.projects.utils: === Fetching project from https://github.com/PacktPublishing/Machine-Learning-Engineering-with-MLflow/tree/master/Chapter01/stockpred into C:\Users\Haps\AppData\Local\Temp\tmpawr8l5_r ===
Traceback (most recent call last):
  File "C:\Users\Haps\anaconda3\envs\mlflow_env\Scripts\mlflow-script.py", line 10, in <module>
    sys.exit(cli())
  File "C:\Users\Haps\anaconda3\envs\mlflow_env\lib\site-packages\click\core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\Haps\anaconda3\envs\mlflow_env\lib\site-packages\click\core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "C:\Users\Haps\anaconda3\envs\mlflow_env\lib\site-packages\click\core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\Haps\anaconda3\envs\mlflow_env\lib\site-packages\click\core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\Haps\anaconda3\envs\mlflow_env\lib\site-packages\click\core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "C:\Users\Haps\anaconda3\envs\mlflow_env\lib\site-packages\mlflow\cli.py", line 181, in run
    run_id=run_id,
  File "C:\Users\Haps\anaconda3\envs\mlflow_env\lib\site-packages\mlflow\projects\__init__.py", line 304, in run
    synchronous=synchronous,
  File "C:\Users\Haps\anaconda3\envs\mlflow_env\lib\site-packages\mlflow\projects\__init__.py", line 99, in _run
    experiment_id,
  File "C:\Users\Haps\anaconda3\envs\mlflow_env\lib\site-packages\mlflow\projects\backend\local.py", line 45, in run
    work_dir = fetch_and_validate_project(project_uri, version, entry_point, params)
  File "C:\Users\Haps\anaconda3\envs\mlflow_env\lib\site-packages\mlflow\projects\utils.py", line 124, in fetch_and_validate_project
    work_dir = _fetch_project(uri=uri, version=version)
  File "C:\Users\Haps\anaconda3\envs\mlflow_env\lib\site-packages\mlflow\projects\utils.py", line 158, in _fetch_project
    _fetch_git_repo(parsed_uri, version, dst_dir)
  File "C:\Users\Haps\anaconda3\envs\mlflow_env\lib\site-packages\mlflow\projects\utils.py", line 185, in _fetch_git_repo
    origin.fetch()
  File "C:\Users\Haps\anaconda3\envs\mlflow_env\lib\site-packages\git\remote.py", line 868, in fetch
    kill_after_timeout=kill_after_timeout)
  File "C:\Users\Haps\anaconda3\envs\mlflow_env\lib\site-packages\git\remote.py", line 732, in _get_fetch_info_from_stderr
    proc.wait(stderr=stderr_text)
  File "C:\Users\Haps\anaconda3\envs\mlflow_env\lib\site-packages\git\cmd.py", line 501, in wait
    raise GitCommandError(remove_password_if_present(self.args), status, errstr)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git fetch -v origin
  stderr: 'fatal: repository 'https://github.com/PacktPublishing/Machine-Learning-Engineering-with-MLflow/tree/master/Chapter01/stockpred/' not found'

This is particularly strange because pulling from other repositories works without a problem.

E.g.)
The following line works perfectly. It runs to completion and creates the logged values in the mlruns/ directory as expected.
mlflow run https://github.com/mlflow/mlflow-example.git -P alpha=5.0

Returning the following:

2021/11/04 14:15:57 INFO mlflow.projects.utils: === Fetching project from https://github.com/mlflow/mlflow-example.git into C:\Users\Haps\AppData\Local\Temp\tmpycy7kden ===
2021/11/04 14:16:02 INFO mlflow.projects.utils: === Created directory C:\Users\Haps\AppData\Local\Temp\tmpttkcntxv for downloading remote URIs passed to arguments of type 'path' ===
2021/11/04 14:16:02 INFO mlflow.projects.backend.local: === Running command 'conda activate mlflow-1abc00771765dd9dd15731cbda4938c765fbb90b && python train.py 0.4 0.1' in run with ID 'a93fafe1cda649de9012f7d069bb6e56' ===
C:\Users\Haps\anaconda3\envs\mlflow-1abc00771765dd9dd15731cbda4938c765fbb90b\lib\site-packages\sklearn\utils\__init__.py:4: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
  from collections import Sequence
C:\Users\Haps\anaconda3\envs\mlflow-1abc00771765dd9dd15731cbda4938c765fbb90b\lib\site-packages\sklearn\model_selection\_split.py:18: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
  from collections import Iterable
C:\Users\Haps\anaconda3\envs\mlflow-1abc00771765dd9dd15731cbda4938c765fbb90b\lib\site-packages\sklearn\model_selection\_search.py:16: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
  from collections import Mapping, namedtuple, defaultdict, Sequence
Elasticnet model (alpha=0.400000, l1_ratio=0.100000):
  RMSE: 0.7909069124367867
  MAE: 0.6174288492244517
  R2: 0.19207580388574486
2021/11/04 14:16:10 INFO mlflow.projects: === Run (ID 'a93fafe1cda649de9012f7d069bb6e56') succeeded ===

[nlauchande]

@hdotking: Thanks for reporting. I can confirm this has an issue, but i recall testing this multiple time before .

Currently digging.

[hdotking]

Thanks so much @nlauchande. I'd really like to get into your book but this bug leaves me stumped at the first Chapter 😢

I've also tried installing mlflow and docker with pip in a virtual env (with out conda) as well as installing mlflow and docker in my default python environment.

They all return the same error. Hope that helps!

[nlauchande]

@hdotking: I have noticed an issue with the way the command was transcribed on that section of the chapter, it should be :
mlflow run https://github.com/PacktPublishing/Machine-Learning-Engineering-with-MLflow.git#Chapter01/stockpred

Unrelated issue:
I noticed another issue with an upstream dependency that I am busy fixing, an issue that started after sending the book for publication: https://stackoverflow.com/questions/68327968/unable-to-get-stock-data-from-yahoo-with-pandas-datareader/68435757#68435757 .

[hdotking]

@nlauchande thanks for the update!

Unfortunately it still does not work.

Here are the steps i've taken before running the new command you provided:

• I am now rerunning this on WSL for consistency
• I uninstalled the existing version of WSL and installed it from scratch
• installed miniconda to access python 3.9 and used the pip install mlflow==1.18 command as described in page 10. (not using a conda env)
• I reinstalled docker and ensured it was configured correctly to WSL2 according to this link https://docs.docker.com/desktop/windows/wsl/

The error message is as follows:

$ mlflow run https://github.com/PacktPublishing/Machine-Learning-Engineering-with-MLflow.git#Chapter01/stockpred
2021/11/07 23:21:38 INFO mlflow.projects.utils: === Fetching project from https://github.com/PacktPublishing/Machine-Learning-Engineering-with-MLflow.git#Chapter01/stockpred into /tmp/tmpmad51fsj ===
2021/11/07 23:21:40 INFO mlflow.projects.docker: === Building docker image stockpred:c8bf265 ===
Traceback (most recent call last):
  File "/home/hdot/miniconda3/bin/mlflow", line 8, in <module>
    sys.exit(cli())
  File "/home/hdot/miniconda3/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/hdot/miniconda3/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/hdot/miniconda3/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/hdot/miniconda3/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/hdot/miniconda3/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/hdot/miniconda3/lib/python3.9/site-packages/mlflow/cli.py", line 168, in run
    projects.run(
  File "/home/hdot/miniconda3/lib/python3.9/site-packages/mlflow/projects/__init__.py", line 293, in run
    submitted_run_obj = _run(
  File "/home/hdot/miniconda3/lib/python3.9/site-packages/mlflow/projects/__init__.py", line 92, in _run
    submitted_run = backend.run(
  File "/home/hdot/miniconda3/lib/python3.9/site-packages/mlflow/projects/backend/local.py", line 72, in run
    image = build_docker_image(
  File "/home/hdot/miniconda3/lib/python3.9/site-packages/mlflow/projects/docker.py", line 69, in build_docker_image
    image, _ = client.images.build(
  File "/home/hdot/miniconda3/lib/python3.9/site-packages/docker/models/images.py", line 285, in build
    raise BuildError(chunk['error'], result_stream)
docker.errors.BuildError: pull access denied for stockpred, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

Many of the answers on stackoverflow point towards a typographic error hidden somewhere in the docker image/File.

Any thoughts?

[nlauchande]

@hdotking : Most of this issues are addressed in here : #15 , i am busy reviewing the changes made by @webmakaka #19 .