compute principal graph calculation struck at node 43 while using multiple cores(n_cores>1)

[kunamneni117]

Hi,
While using python implementation of elpigraph leveraging n_cores>1,the computeprincipalgraph calculation struck at Node 43 .Below attached the code output for reference.

Computing EPG with 60 nodes on 199 points and 3 dimensions
Creating a Pool with 2 processes
Nodes = 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43
I have used the tree data which is in the same repository for doing sanity checks.

In the library I have removed the commentated part multiprocessing library for using multi cores.

[j-bac]

Thanks for the report! Using the example tree data, which has 492 points, I find no problem when uncommenting import multiprocessing as mp in BaseElpi.py and grammar_operations.py. You seem to have used a different dataset with 199 points and I can't reproduce your issue.

I disabled the multi-cpu option for now because it created some issues when used in clusters. If you wish to try it yourself, to see any speedup you need a dataset with ~10**4 points minimum. A better option to speed up your code is to try the GPU version (import elpigraphgpu) using your own GPU or a free one on google Colaboratory.

[kunamneni117]

Thanks for the prompt response.I have tried again with uncommenting the import multiprocessing as mp in BaseElpi.py and grammar_operations.py.
I have used the tree data set. Earlier i have used circle data with header on.(199 data points in the earlier comment)
Generating the initial configuration
Creating a chain in the 1st PC with 2 nodes
90% of the points have been used as initial conditions. Resetting.
Constructing tree 1 of 1 / Subset 1 of 1
Performing PCA
Using standard PCA
3 dimensions are being used
100.0 % of the original variance has been retained
The elastic matrix is being used. Edge configuration will be ignored
Computing EPG with 60 nodes on 492 points and 3 dimensions
Creating a Pool with 4 processes
Nodes = 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43

If you can provide some tips how to debug the issue i can provide more info about the issue.

I am doing checks on multi core approach b/w python and elpigraph implementation.
I am going to use the above elpigraph implementation lot of data(more than 1 million).So i am trying to speed up the process with multi core approach and GPU approach.

The above issue(Multi core issue) can be due to some issue in environment setup? any chance?

[j-bac]

Yes, if you provide more info I am happy to help - however I am quite puzzled what could be causing this, on my laptop with Ubuntu 18.04.4 LTS I have no issue.

In any case the multi-core version is still work in progress and I strongly recommend you do not use it but instead try the GPU version. For 1 million+ datapoints it is the best solution. Using it requires the cupy package, after that using it is as easy as "import elpigraphgpu" instead of "import elpigraph".

Here are example timings:

[kunamneni117]

I tried elpigraph in Ubuntu 18.04 LTS and surprisingly it is not struck at node 43 while doing principal graph computation as u called out.I tried with windows again and computation of principal graph struck a Node 43.I have used the tree data while checking.
I am struck at how to debug the issue further so that i can solve/report the issue.If can u provide some info where to start for debugging the issue that may be useful for me to proceed further.

[j-bac]

I found this link which might be useful

[kunamneni117]

I found the python implementation using multi cores works for small data sets without any issue but when using a medium data(50k data points) hangs while doing computation in Ubuntu 18.04 LTS OS.

Used Supermarket data for testing where used the Order.ID,Order.Date,Ship.Date,Ship.Mode,Customer.ID,Customer.Name,Segment,City,State,Country,Market,Region,Product.ID,Category,Sub.Category,Product.Name,Sales,Quantity,Discount,Profit,Shipping.Cost,Order.Priority columns where some of the columns are Label Encoded.

I have forked your R implementation of elpigraph and its working fine with the above mentioned data in single core and multi core approach.

[j-bac]

I hope to set up a new version for multi-core Python soon and will look into this if I access a windows machine.
In the meantime if you wish to use a multi-core and not GPU implementation it is indeed best to use the R version. Glad to hear it works!

[kunamneni117]

I used tree data for checking multi core issue,got some interesting bugs..At Node 43 it is calling again the total code.
USED WINDOWS MACHINE
Sample Code

import warnings
warnings.filterwarnings('ignore')
import os
import pandas as pd
from elpigraph import computeElasticPrincipalCurve
file_location=r'C:\Users\tonystark\Documents\folder\elpigraph\testing\data'
file_lists=[]
for r,d,f in os.walk(file_location):
for file in f:
if '.csv' in file:
file_lists.append(os.path.join(r, file))
file_name=file_lists[2]
print("Generating ElPiGraph output for file -",file_name + os.linesep)
first_output = pd.read_csv(file_name,header=None,na_values=["NA","na","NULL","null",""," "])
print("Computing Elastic Principal Curve for file-",file_name + os.linesep)
computeElasticPrincipalCurve(first_output.astype('float64').values, 60,n_cores=4)

C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\python.exe "C:/Users/tonystark/Documents/folder/elpigraph/testing/mutliprocessing check.py"
Generating ElPiGraph output for file - C:\Users\tonystark\Documents\folder\elpigraph\testing\data\tree_data.csv

Computing Elastic Principal Curve for file- C:\Users\tonystark\Documents\folder\elpigraph\testing\data\tree_data.csv

Generating the initial configuration
Creating a chain in the 1st PC with 2 nodes
90% of the points have been used as initial conditions. Resetting.
Constructing tree 1 of 1 / Subset 1 of 1
Performing PCA
Using standard PCA
3 dimensions are being used
100.0 % of the original variance has been retained
The elastic matrix is being used. Edge configuration will be ignored
Computing EPG with 60 nodes on 492 points and 3 dimensions
Creating a Pool with 4 processes
Nodes = 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43
Generating ElPiGraph output for file - C:\Users\tonystark\Documents\folder\elpigraph\testing\data\tree_data.csv

Computing Elastic Principal Curve for file- C:\Users\tonystark\Documents\folder\elpigraph\testing\data\tree_data.csv

Generating the initial configuration
Creating a chain in the 1st PC with 2 nodes
Generating ElPiGraph output for file - C:\Users\tonystark\Documents\folder\elpigraph\testing\data\tree_data.csv

Computing Elastic Principal Curve for file- C:\Users\tonystark\Documents\folder\elpigraph\testing\data\tree_data.csv

Generating the initial configuration
Creating a chain in the 1st PC with 2 nodes
Generating ElPiGraph output for file - C:\Users\tonystark\Documents\folder\elpigraph\testing\data\tree_data.csv

Computing Elastic Principal Curve for file- C:\Users\tonystark\Documents\folder\elpigraph\testing\data\tree_data.csv

Generating the initial configuration
Creating a chain in the 1st PC with 2 nodes
Generating ElPiGraph output for file - C:\Users\tonystark\Documents\folder\elpigraph\testing\data\tree_data.csv

Computing Elastic Principal Curve for file- C:\Users\tonystark\Documents\folder\elpigraph\testing\data\tree_data.csv

Generating the initial configuration
Creating a chain in the 1st PC with 2 nodes
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 105, in spawn_main
exitcode = _main(fd)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 114, in _main
prepare(preparation_data)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 225, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
run_name="mp_main")
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 263, in run_path
Traceback (most recent call last):
File "", line 1, in
pkg_name=pkg_name, script_name=fname)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\tonystark\Documents\folder\elpigraph\testing\mutliprocessing check.py", line 24, in
computeElasticPrincipalCurve(first_output.astype('float64').values, 60,n_cores=4)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph_topologies.py", line 597, in computeElasticPrincipalCurve
MaxSteps = MaxSteps
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph_BaseElPiWrapper.py", line 284, in computeElasticPrincipalGraphWithGrammars
DisplayWarnings=DisplayWarnings
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph\src\BaseElPi.py", line 585, in computeElasticPrincipalGraph
n_cores=n_cores,MinParOp=MinParOp)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph\src\BaseElPi.py", line 184, in ElPrincGraph
pool = mp.Pool(n_cores)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\context.py", line 119, in Pool
Traceback (most recent call last):
File "", line 1, in
context=self.get_context())
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 177, in init
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 105, in spawn_main
self._repopulate_pool()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 238, in _repopulate_pool
self._wrap_exception)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 257, in _repopulate_pool_static
w.start()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\process.py", line 112, in start
exitcode = _main(fd)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 114, in _main
prepare(preparation_data)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 225, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
run_name="mp_main")
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\tonystark\Documents\folder\elpigraph\testing\mutliprocessing check.py", line 24, in
self._popen = self._Popen(self)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\popen_spawn_win32.py", line 33, in init
computeElasticPrincipalCurve(first_output.astype('float64').values, 60,n_cores=4)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph_topologies.py", line 597, in computeElasticPrincipalCurve
MaxSteps = MaxSteps
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph_BaseElPiWrapper.py", line 284, in computeElasticPrincipalGraphWithGrammars
DisplayWarnings=DisplayWarnings
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph\src\BaseElPi.py", line 585, in computeElasticPrincipalGraph
prep_data = spawn.get_preparation_data(process_obj._name)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
_check_not_importing_main()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
is not going to be frozen to produce an executable.''')
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.
n_cores=n_cores,MinParOp=MinParOp)

File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph\src\BaseElPi.py", line 184, in ElPrincGraph
pool = mp.Pool(n_cores)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\context.py", line 119, in Pool
context=self.get_context())
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 177, in init
self._repopulate_pool()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 238, in _repopulate_pool
self._wrap_exception)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 257, in _repopulate_pool_static
w.start()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\process.py", line 112, in start
self._popen = self._Popen(self)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\popen_spawn_win32.py", line 33, in init
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 105, in spawn_main
prep_data = spawn.get_preparation_data(process_obj._name)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
_check_not_importing_main()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
is not going to be frozen to produce an executable.''')
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.
exitcode = _main(fd)

File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 114, in _main
prepare(preparation_data)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 225, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
run_name="mp_main")
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\tonystark\Documents\folder\elpigraph\testing\mutliprocessing check.py", line 24, in
computeElasticPrincipalCurve(first_output.astype('float64').values, 60,n_cores=4)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph_topologies.py", line 597, in computeElasticPrincipalCurve
MaxSteps = MaxSteps
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph_BaseElPiWrapper.py", line 284, in computeElasticPrincipalGraphWithGrammars
DisplayWarnings=DisplayWarnings
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph\src\BaseElPi.py", line 585, in computeElasticPrincipalGraph
n_cores=n_cores,MinParOp=MinParOp)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph\src\BaseElPi.py", line 184, in ElPrincGraph
pool = mp.Pool(n_cores)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\context.py", line 119, in Pool
context=self.get_context())
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 177, in init
self._repopulate_pool()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 238, in _repopulate_pool
self._wrap_exception)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 257, in _repopulate_pool_static
w.start()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\process.py", line 112, in start
self._popen = self._Popen(self)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\popen_spawn_win32.py", line 33, in init
prep_data = spawn.get_preparation_data(process_obj._name)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
_check_not_importing_main()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
is not going to be frozen to produce an executable.''')
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

90% of the points have been used as initial conditions. Resetting.
Constructing tree 1 of 1 / Subset 1 of 1
Performing PCA
Using standard PCA
3 dimensions are being used
100.0 % of the original variance has been retained
The elastic matrix is being used. Edge configuration will be ignored
Computing EPG with 60 nodes on 492 points and 3 dimensions
Creating a Pool with 4 processes
90% of the points have been used as initial conditions. Resetting.
90% of the points have been used as initial conditions. Resetting.
Constructing tree 1 of 1 / Subset 1 of 1
Performing PCA
Using standard PCA
3 dimensions are being used
100.0 % of the original variance has been retained
The elastic matrix is being used. Edge configuration will be ignored
Computing EPG with 60 nodes on 492 points and 3 dimensions
Creating a Pool with 4 processes
Constructing tree 1 of 1 / Subset 1 of 1
Performing PCA
Using standard PCA
3 dimensions are being used
100.0 % of the original variance has been retained
The elastic matrix is being used. Edge configuration will be ignored
Computing EPG with 60 nodes on 492 points and 3 dimensions
Creating a Pool with 4 processes
90% of the points have been used as initial conditions. Resetting.
Constructing tree 1 of 1 / Subset 1 of 1
Performing PCA
Using standard PCA
3 dimensions are being used
100.0 % of the original variance has been retained
The elastic matrix is being used. Edge configuration will be ignored
Computing EPG with 60 nodes on 492 points and 3 dimensions
Creating a Pool with 4 processes
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 105, in spawn_main
exitcode = _main(fd)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 114, in _main
prepare(preparation_data)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 225, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
run_name="mp_main")
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\tonystark\Documents\folder\elpigraph\testing\mutliprocessing check.py", line 24, in
computeElasticPrincipalCurve(first_output.astype('float64').values, 60,n_cores=4)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph_topologies.py", line 597, in computeElasticPrincipalCurve
MaxSteps = MaxSteps
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph_BaseElPiWrapper.py", line 284, in computeElasticPrincipalGraphWithGrammars
DisplayWarnings=DisplayWarnings
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph\src\BaseElPi.py", line 585, in computeElasticPrincipalGraph
n_cores=n_cores,MinParOp=MinParOp)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph\src\BaseElPi.py", line 184, in ElPrincGraph
pool = mp.Pool(n_cores)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\context.py", line 119, in Pool
context=self.get_context())
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 177, in init
self._repopulate_pool()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 238, in _repopulate_pool
self._wrap_exception)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 257, in _repopulate_pool_static
w.start()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\process.py", line 112, in start
self._popen = self._Popen(self)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\popen_spawn_win32.py", line 33, in init
prep_data = spawn.get_preparation_data(process_obj._name)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
_check_not_importing_main()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
is not going to be frozen to produce an executable.''')
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

Generating ElPiGraph output for file - C:\Users\tonystark\Documents\folder\elpigraph\testing\data\tree_data.csv

Computing Elastic Principal Curve for file- C:\Users\tonystark\Documents\folder\elpigraph\testing\data\tree_data.csv

Generating the initial configuration
Creating a chain in the 1st PC with 2 nodes
Generating ElPiGraph output for file - C:\Users\tonystark\Documents\folder\elpigraph\testing\data\tree_data.csv

Computing Elastic Principal Curve for file- C:\Users\tonystark\Documents\folder\elpigraph\testing\data\tree_data.csv

Generating the initial configuration
Creating a chain in the 1st PC with 2 nodes
Generating ElPiGraph output for file - C:\Users\tonystark\Documents\folder\elpigraph\testing\data\tree_data.csv

Computing Elastic Principal Curve for file- C:\Users\tonystark\Documents\folder\elpigraph\testing\data\tree_data.csv

Generating the initial configuration
Creating a chain in the 1st PC with 2 nodes
Generating ElPiGraph output for file - C:\Users\tonystark\Documents\folder\elpigraph\testing\data\tree_data.csv

Computing Elastic Principal Curve for file- C:\Users\tonystark\Documents\folder\elpigraph\testing\data\tree_data.csv

Generating the initial configuration
Creating a chain in the 1st PC with 2 nodes
90% of the points have been used as initial conditions. Resetting.
90% of the points have been used as initial conditions. Resetting.
Constructing tree 1 of 1 / Subset 1 of 1
Performing PCA
Using standard PCA
3 dimensions are being used
100.0 % of the original variance has been retained
The elastic matrix is being used. Edge configuration will be ignored
Computing EPG with 60 nodes on 492 points and 3 dimensions
Traceback (most recent call last):
File "", line 1, in
Creating a Pool with 4 processes
Constructing tree 1 of 1 / Subset 1 of 1
Performing PCA
Using standard PCA
3 dimensions are being used
100.0 % of the original variance has been retained
The elastic matrix is being used. Edge configuration will be ignored
Computing EPG with 60 nodes on 492 points and 3 dimensions
Creating a Pool with 4 processes
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 105, in spawn_main
exitcode = _main(fd)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 114, in _main
prepare(preparation_data)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 225, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
run_name="mp_main")
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 263, in run_path
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 105, in spawn_main
pkg_name=pkg_name, script_name=fname)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\tonystark\Documents\folder\elpigraph\testing\mutliprocessing check.py", line 24, in
exitcode = _main(fd)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 114, in _main
prepare(preparation_data)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 225, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
run_name="mp_main")
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 263, in run_path
computeElasticPrincipalCurve(first_output.astype('float64').values, 60,n_cores=4)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph_topologies.py", line 597, in computeElasticPrincipalCurve
MaxSteps = MaxSteps
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph_BaseElPiWrapper.py", line 284, in computeElasticPrincipalGraphWithGrammars
DisplayWarnings=DisplayWarnings
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph\src\BaseElPi.py", line 585, in computeElasticPrincipalGraph
pkg_name=pkg_name, script_name=fname)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\tonystark\Documents\folder\elpigraph\testing\mutliprocessing check.py", line 24, in
computeElasticPrincipalCurve(first_output.astype('float64').values, 60,n_cores=4)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph_topologies.py", line 597, in computeElasticPrincipalCurve
MaxSteps = MaxSteps
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph_BaseElPiWrapper.py", line 284, in computeElasticPrincipalGraphWithGrammars
DisplayWarnings=DisplayWarnings
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph\src\BaseElPi.py", line 585, in computeElasticPrincipalGraph
n_cores=n_cores,MinParOp=MinParOp)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph\src\BaseElPi.py", line 184, in ElPrincGraph
pool = mp.Pool(n_cores)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\context.py", line 119, in Pool
n_cores=n_cores,MinParOp=MinParOp)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph\src\BaseElPi.py", line 184, in ElPrincGraph
pool = mp.Pool(n_cores)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\context.py", line 119, in Pool
context=self.get_context())
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 177, in init
context=self.get_context())
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 177, in init
self._repopulate_pool()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 238, in _repopulate_pool
self._wrap_exception)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 257, in _repopulate_pool_static
w.start()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\process.py", line 112, in start
self._repopulate_pool()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 238, in _repopulate_pool
self._wrap_exception)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 257, in _repopulate_pool_static
w.start()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\process.py", line 112, in start
self._popen = self._Popen(self)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\popen_spawn_win32.py", line 33, in init
self._popen = self._Popen(self)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\popen_spawn_win32.py", line 33, in init
prep_data = spawn.get_preparation_data(process_obj._name)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
_check_not_importing_main()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
is not going to be frozen to produce an executable.''')
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.
prep_data = spawn.get_preparation_data(process_obj._name)

File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
_check_not_importing_main()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
is not going to be frozen to produce an executable.''')
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

90% of the points have been used as initial conditions. Resetting.
Constructing tree 1 of 1 / Subset 1 of 1
Performing PCA
Using standard PCA
3 dimensions are being used
100.0 % of the original variance has been retained
The elastic matrix is being used. Edge configuration will be ignored
Computing EPG with 60 nodes on 492 points and 3 dimensions
Creating a Pool with 4 processes
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 105, in spawn_main
exitcode = _main(fd)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 114, in _main
prepare(preparation_data)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 225, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
run_name="mp_main")
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\tonystark\Documents\folder\elpigraph\testing\mutliprocessing check.py", line 24, in
computeElasticPrincipalCurve(first_output.astype('float64').values, 60,n_cores=4)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph_topologies.py", line 597, in computeElasticPrincipalCurve
MaxSteps = MaxSteps
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph_BaseElPiWrapper.py", line 284, in computeElasticPrincipalGraphWithGrammars
DisplayWarnings=DisplayWarnings
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph\src\BaseElPi.py", line 585, in computeElasticPrincipalGraph
n_cores=n_cores,MinParOp=MinParOp)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph\src\BaseElPi.py", line 184, in ElPrincGraph
pool = mp.Pool(n_cores)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\context.py", line 119, in Pool
context=self.get_context())
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 177, in init
self._repopulate_pool()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 238, in _repopulate_pool
self._wrap_exception)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 257, in _repopulate_pool_static
w.start()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\process.py", line 112, in start
self._popen = self._Popen(self)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\popen_spawn_win32.py", line 33, in init
prep_data = spawn.get_preparation_data(process_obj._name)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
_check_not_importing_main()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
is not going to be frozen to produce an executable.''')
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

90% of the points have been used as initial conditions. Resetting.
Constructing tree 1 of 1 / Subset 1 of 1
Performing PCA
Using standard PCA
3 dimensions are being used
100.0 % of the original variance has been retained
The elastic matrix is being used. Edge configuration will be ignored
Computing EPG with 60 nodes on 492 points and 3 dimensions
Creating a Pool with 4 processes
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 105, in spawn_main
exitcode = _main(fd)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 114, in _main
prepare(preparation_data)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 225, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
run_name="mp_main")
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\tonystark\Documents\folder\elpigraph\testing\mutliprocessing check.py", line 24, in
computeElasticPrincipalCurve(first_output.astype('float64').values, 60,n_cores=4)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph_topologies.py", line 597, in computeElasticPrincipalCurve
MaxSteps = MaxSteps
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph_BaseElPiWrapper.py", line 284, in computeElasticPrincipalGraphWithGrammars
DisplayWarnings=DisplayWarnings
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph\src\BaseElPi.py", line 585, in computeElasticPrincipalGraph
n_cores=n_cores,MinParOp=MinParOp)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\site-packages\elpigraph\src\BaseElPi.py", line 184, in ElPrincGraph
pool = mp.Pool(n_cores)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\context.py", line 119, in Pool
context=self.get_context())
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 177, in init
self._repopulate_pool()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 238, in _repopulate_pool
self._wrap_exception)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\pool.py", line 257, in _repopulate_pool_static
w.start()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\process.py", line 112, in start
self._popen = self._Popen(self)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\popen_spawn_win32.py", line 33, in init
prep_data = spawn.get_preparation_data(process_obj._name)
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
_check_not_importing_main()
File "C:\Users\tonystark\AppData\Local\conda\conda\envs\myenv\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
is not going to be frozen to produce an executable.''')
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

[kunamneni117]

I think link give some light for the current issue.