Core-4.0.0 w/ ply-3.6 bombs at bake init.

[mikaelj]

Decided to do a fresh build after upgrading to core/4.0.0 by removing tmp/.

huvudfoting [11:07] /data/sabre/oelite/tlabos> oe bake -d lpc21isp
whitelist=['PATH', 'PWD', 'SHELL', 'TERM', 'MACHINE', 'DISTRO']
Whitelist filtered shell environment:
> PATH=/home/mikaelj/local/bin:/home/mikaelj/local/sbin:/opt/android-sdk-linux/tools:/opt/android-sdk-linux/platform-tools:/home/mikaelj/local/bin:/home/mikaelj/local/sbin:/opt/android-sdk-linux/tools:/opt/android-sdk-linux/platform-tools:/home/mikaelj/local/bin:/home/mikaelj/local/sbin:/opt/android-sdk-linux/tools:/opt/android-sdk-linux/platform-tools:/opt/arm-cs-tools/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/mikaelj/.local/bin:/home/mikaelj/.local/bin:/home/mikaelj/.local/bin/:/home/mikaelj/.local/bin:/home/mikaelj/.local/bin/
> PWD=/data/sabre/oelite/tlabos
> SHELL=bash
> TERM=screen-256color-s
ERROR: exception in bake.run()

Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/oebakery/cmd/cmds.py", line 137, in call
    ret = function(*args)
  File "/data/sabre/oelite/tlabos/meta/core/lib/oelite/cmd/bake.py", line 29, in run
    baker = oelite.baker.OEliteBaker(options, args, config)
  File "/data/sabre/oelite/tlabos/meta/core/lib/oelite/baker.py", line 115, in __init__
    self.confparser = confparse.ConfParser(self.config)
  File "/data/sabre/oelite/tlabos/meta/core/lib/oelite/parse/confparse.py", line 10, in __init__
    super(ConfParser, self).__init__(data, parent, **kwargs)
  File "/data/sabre/oelite/tlabos/meta/core/lib/oelite/parse/oeparse.py", line 26, in __init__
    self.yacc = ply.yacc.yacc(module=self, debug=0, picklefile=picklefile)
  File "/usr/local/lib/python2.7/dist-packages/ply-3.6-py2.7.egg/ply/yacc.py", line 3242, in yacc
    read_signature = lr.read_pickle(picklefile)
  File "/usr/local/lib/python2.7/dist-packages/ply-3.6-py2.7.egg/ply/yacc.py", line 1986, in read_pickle
    in_f = open(filename, 'rb')
IOError: [Errno 2] No such file or directory: 'tmp/ply/oelite.parse.confparse.p'

CRITICAL: bake failed: Exception: [Errno 2] No such file or directory: 'tmp/ply/oelite.parse.confparse.p'
huvudfoting [11:07] /data/sabre/oelite/tlabos -1-> 

tmp/ply is an empty directory.

[mikaelj]

"workaround":

diff --git a/lib/oelite/parse/oeparse.py b/lib/oelite/parse/oeparse.py
index d2087a9..7ccbdb7 100644
--- a/lib/oelite/parse/oeparse.py
+++ b/lib/oelite/parse/oeparse.py
@@ -22,7 +22,8 @@ class OEParser(object):
             # ply <= 3.4
             self.tokens = lexer.lextokens.keys()
         oelite.util.makedirs("tmp/ply")
-        picklefile = "tmp/ply/" + self.__class__.__module__ + ".p"
+        #picklefile = "tmp/ply/" + self.__class__.__module__ + ".p"
+        picklefile = None
         self.yacc = ply.yacc.yacc(module=self, debug=0, picklefile=picklefile)
         if meta is not None:
             self.meta = meta

As far as I can tell, the code in both ply-3.4 and core-3.5 (my previous versions) were the same in the relevant places, i.e. ply.yacc.yacc(..., picklefile=picklefile) and read_pickle().

[esben]

Hmm... Maybe we should apply this workaround for now.

I assume it will degrade parsing performance considerably, but better slow and correct, than fail badly in high speed.

[mikaelj]

I've seen a file being created that looks ply-ish in meta/core/... somewhere (can look later if you want to know), so it's only slow on first build.

[esben]

Have you had time to look more into it.

Would be nice to resolve this issue before releasing 4.0.1.

[mikaelj]

No, sorry, I haven't.

What I have noticed is that after certain recipe changes (haven't found a
pattern), all recipes need be reparsed, but usally it takes the regular
amount fo time.

This because the data is still cached, just in another location:
core/lib/oelite/parse/parsetab.py

On Mon, Oct 12, 2015 at 1:13 PM, Esben Haabendal notifications@github.com
wrote:

Have you had time to look more into it.

Would be nice to resolve this issue before releasing 4.0.1.

—
Reply to this email directly or view it on GitHub
#30 (comment).