Failing test cases after building

[jasonhemann]

Following the README instructions, I run the provided unit tests, and python reports two failing tests. Can you reproduce this behavior?

.......FF.............
======================================================================
FAIL: test_violated_abstract (tyrell.decider.test_example_constraint.TestExampleConstraint)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/me/Trinity/tyrell/decider/test_example_constraint.py", line 80, in test_violated_abstract
    self.assertIn([(prog, prog.production)], reason)
AssertionError: [(ApplyNode(mult, [ParamNode(0), ParamNode(1)]), FunctionProduction(id=0, lhs=ValueType(IntExpr, properties={'pos': <ExprType.BOOL: 'bool'>, 'neg': <ExprType.BOOL: 'bool'>}), name=mult, rhs=[ValueType(IntExpr, properties={'pos': <ExprType.BOOL: 'bool'>, 'neg': <ExprType.BOOL: 'bool'>}), ValueType(IntExpr, properties={'pos': <ExprType.BOOL: 'bool'>, 'neg': <ExprType.BOOL: 'bool'>})]))] not found in [[Blame(node=@param0, production=2), Blame(node=@param1, production=3), Blame(node=mult(@param0, @param1), production=1)], [(ParamNode(0), ParamProduction(id=2, lhs=ValueType(IntExpr, properties={'pos': <ExprType.BOOL: 'bool'>, 'neg': <ExprType.BOOL: 'bool'>}), param_id=0)), (ParamNode(1), ParamProduction(id=3, lhs=ValueType(IntExpr, properties={'pos': <ExprType.BOOL: 'bool'>, 'neg': <ExprType.BOOL: 'bool'>}), param_id=1)), (ApplyNode(mult, [ParamNode(0), ParamNode(1)]), FunctionProduction(id=0, lhs=ValueType(IntExpr, properties={'pos': <ExprType.BOOL: 'bool'>, 'neg': <ExprType.BOOL: 'bool'>}), name=mult, rhs=[ValueType(IntExpr, properties={'pos': <ExprType.BOOL: 'bool'>, 'neg': <ExprType.BOOL: 'bool'>}), ValueType(IntExpr, properties={'pos': <ExprType.BOOL: 'bool'>, 'neg': <ExprType.BOOL: 'bool'>})]))]]

======================================================================
FAIL: test_violated_abstract2 (tyrell.decider.test_example_constraint.TestExampleConstraint)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/me/Trinity/tyrell/decider/test_example_constraint.py", line 95, in test_violated_abstract2
    self.assertIn([(prog, prog.production)], reason)
AssertionError: [(ApplyNode(div, [ParamNode(0), ParamNode(1)]), FunctionProduction(id=1, lhs=ValueType(IntExpr, properties={'pos': <ExprType.BOOL: 'bool'>, 'neg': <ExprType.BOOL: 'bool'>}), name=div, rhs=[ValueType(IntExpr, properties={'pos': <ExprType.BOOL: 'bool'>, 'neg': <ExprType.BOOL: 'bool'>}), ValueType(IntExpr, properties={'pos': <ExprType.BOOL: 'bool'>, 'neg': <ExprType.BOOL: 'bool'>})]))] not found in [[Blame(node=@param0, production=2), Blame(node=@param1, production=3), Blame(node=div(@param0, @param1), production=0)], [(ParamNode(0), ParamProduction(id=2, lhs=ValueType(IntExpr, properties={'pos': <ExprType.BOOL: 'bool'>, 'neg': <ExprType.BOOL: 'bool'>}), param_id=0)), (ApplyNode(div, [ParamNode(0), ParamNode(1)]), FunctionProduction(id=1, lhs=ValueType(IntExpr, properties={'pos': <ExprType.BOOL: 'bool'>, 'neg': <ExprType.BOOL: 'bool'>}), name=div, rhs=[ValueType(IntExpr, properties={'pos': <ExprType.BOOL: 'bool'>, 'neg': <ExprType.BOOL: 'bool'>}), ValueType(IntExpr, properties={'pos': <ExprType.BOOL: 'bool'>, 'neg': <ExprType.BOOL: 'bool'>})])), (ParamNode(1), ParamProduction(id=3, lhs=ValueType(IntExpr, properties={'pos': <ExprType.BOOL: 'bool'>, 'neg': <ExprType.BOOL: 'bool'>}), param_id=1))]]

----------------------------------------------------------------------
Ran 22 tests in 0.208s

FAILED (failures=2)

I believe I've installed Trinity correctly (caveat: see #10 ). I'm running on OSX, if that's relevant, but I saw the same failures on a RHEL machine.

[zhangxii]

I also ran into this problem and failed on the same two test cases. I wonder if you solved the problem?

[jasonhemann]

No, I've made no progress since then.

[chyanju]

A side note (may not be relevant to this issue but may be helpful if your goal is to run Trinity): You can just clone the repo and directly run the demos from the repo root without installing Trinity.

[jasonhemann]

@chyanju Could you say more? Following those instructions literally---downloading the repository and then invoking, e.g. python3 demo_morpheus_enumerator.py signals an error. What am I misunderstanding?

[chyanju]

@chyanju Could you say more? Following those instructions literally---downloading the repository and then invoking, e.g. python3 demo_morpheus_enumerator.py signals an error. What am I misunderstanding?

Could you paste the error here? It could be missing dependencies...

[jasonhemann]

@chyanju ,

I'm not 100% clear the approach you wanted me to try, so consequently the error message you want to see. So I include the ones I think relevant, and you can let me know if I misunderstand. When I follow (my modified version of, cf #10) the install instructions, I see the error messages at the top of this issue.

When I download the repository, follow those installation instructions up to pip install, and modify the rest to the following

 - pip install wheel sexpdata rpy2 compare 
 - pip install -e ".[dev]"
 - python3 setup.py sdist
 - parse-tyrell-spec example/toy.tyrell
 - python3 demo_morpheus_enumerator.py

I see the following message:

R[write to console]: Error in library(compare) : there is no package called ‘compare’

Traceback (most recent call last):
  File "/Users/jhemann/Trinity/demo_morpheus_enumerator.py", line 16, in <module>
    robjects.r('''
  File "/Users/jhemann/Trinity/tyrell/venv/lib/python3.9/site-packages/rpy2/robjects/__init__.py", line 438, in __call__
    res = self.eval(p)
  File "/Users/jhemann/Trinity/tyrell/venv/lib/python3.9/site-packages/rpy2/robjects/functions.py", line 198, in __call__
    return (super(SignatureTranslatedFunction, self)
  File "/Users/jhemann/Trinity/tyrell/venv/lib/python3.9/site-packages/rpy2/robjects/functions.py", line 125, in __call__
    res = super(Function, self).__call__(*new_args, **new_kwargs)
  File "/Users/jhemann/Trinity/tyrell/venv/lib/python3.9/site-packages/rpy2/rinterface_lib/conversion.py", line 45, in _
    cdata = function(*args, **kwargs)
  File "/Users/jhemann/Trinity/tyrell/venv/lib/python3.9/site-packages/rpy2/rinterface.py", line 680, in __call__
    raise embedded.RRuntimeError(_rinterface._geterrmessage())
rpy2.rinterface_lib.embedded.RRuntimeError: Error in library(compare) : there is no package called ‘compare’

I figured you did not mean literally the following:

1. Download the repository
2. Enter Trinity
3. python3 demo_morpheus_enumerator.py

But I tried it anyway. When I did so I see the following.

Traceback (most recent call last):
  File "/Users/jhemann/Trinity/demo_morpheus_enumerator.py", line 10, in <module>
    import rpy2.robjects as robjects
  File "/usr/local/lib/python3.9/site-packages/rpy2/robjects/__init__.py", line 16, in <module>
    import rpy2.rinterface as rinterface
  File "/usr/local/lib/python3.9/site-packages/rpy2/rinterface.py", line 12, in <module>
    from rpy2.rinterface_lib import openrlib
  File "/usr/local/lib/python3.9/site-packages/rpy2/rinterface_lib/openrlib.py", line 44, in <module>
    rlib = _dlopen_rlib(R_HOME)
  File "/usr/local/lib/python3.9/site-packages/rpy2/rinterface_lib/openrlib.py", line 37, in _dlopen_rlib
    rlib = ffi.dlopen(lib_path)
OSError: cannot load library '/usr/local/Cellar/r/4.1.2/lib/R/lib/libR.dylib': dlopen(/usr/local/Cellar/r/4.1.2/lib/R/lib/libR.dylib, 2): Symbol not found: ___addtf3
  Referenced from: /usr/local/opt/gcc/lib/gcc/11/libquadmath.0.dylib
  Expected in: /usr/lib/libSystem.B.dylib
 in /usr/local/opt/gcc/lib/gcc/11/libquadmath.0.dylib

Is any of that what you wanted me to try?

[chyanju]

@jasonhemann Actually you can just download the repo and run python3 demo_morpheus_enumerator.py (there may be some Python dependency issues which I believe you've already solved by reading the error messages).

For the first error message that you've posted:

R[write to console]: Error in library(compare) : there is no package called ‘compare’

it's corresponding to the R environment. The "compare" library is what the morpheus synthesizer here uses to compare two tables (it's using the "rpy" packages to communicate with external R environment). One solution to this issue is to run your local R REPL, and install the missing packages, for example using the following commands in your R environment:

install.packages("dplyr")
install.packages("tidyr")
install.packages("compare")

and hopefully the first error will be gone.

As for the second error message, it's probably due to the compatibility issue of rpy2, as we also experienced this in some other projects that use rpy2. You can try to downgrade your local rpy2 version to <=3.0.5 and try again to see if this resolves the issue.

Actually since all the errors are caused by the binding of R environment as I can see, instead of trying the python3 demo_morpheus_enumerator.py, if your goal is to build your own synthesizer, you can directly try python3 demo_string_enumerator.py, which will initiate a different synthesizer that gets rid of the R environment so that you can quickly test whether the environment is ready or not.

[jasonhemann]

Ahh, that's very helpful. I didn't read that message carefully enough to notice it was the R compare library. Thank you @chyanju . I did need to install those three R packages. As you suspected even after doing so I still see the following error running python3 demo_morpheus_enumerator.py. However the demo_string_enumerator.py test passes, so my environment must be properly set up.

R[write to console]: 
Attaching package: ‘compare’


R[write to console]: The following object is masked from ‘package:base’:

    Istrue


R[write to console]: 
Attaching package: ‘dplyr’


R[write to console]: The following objects are masked from ‘package:stats’:

    filter, lag


R[write to console]: The following objects are masked from ‘package:base’:

    intersect, setdiff, setequal, union


Traceback (most recent call last):
  File "/Users/jhemann/Trinity/demo_morpheus_enumerator.py", line 371, in <module>
    main()
  File "/Users/jhemann/Trinity/demo_morpheus_enumerator.py", line 336, in main
    depth_val = loc_val + 1
TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

[jasonhemann]

@zhangxii , I believe @chyanju got me sufficiently sorted that this conversation may be of some help to you.

[chyanju]

@jasonhemann No problem. I guess your environment is set up correctly. For the last error message that you've posted:

Traceback (most recent call last):
File "/Users/jhemann/Trinity/demo_morpheus_enumerator.py", line 371, in
main()
File "/Users/jhemann/Trinity/demo_morpheus_enumerator.py", line 336, in main
depth_val = loc_val + 1
TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

I guess that's because the morpheus synthesizer demo_morpheus_enumerator.py takes arguments. For example you should probably call it from the repo root like this:

python3 ./demo_morpheus_enumerator.py --input0 ./benchmarks/pldi17/p3_input1.csv --output ./benchmarks/pldi17/p3_output1.csv --length 3

If the R environment and the rpy2 configuration are correct and compatible (well currently it's probably not according to the previous messages), this synthesizer will also run free of exceptions.

But if it's for environment testing purposes, then I guess running python 3demo_string_enumerator.py should be sufficient enough.