Skip to content

Multiparser occasionally misses final lines of tailed files #155

@wk9874

Description

@wk9874

Describe the bug
Occasionally when tailing a file, Multiparser will seemingly not read the final line(s?) of files.

Expected behavior
All lines from a file which match chosen regex patterns should be reported by tail

Evidence
I have put a print(_data) statement into line 66 of multiparser/parsing/tail.py, which will tell me every line which is being detected by multiparser which matches the tracked values (and which is then passed up to the callback function). I have a file which has a final few lines like this:

 2 Nonlinear |R| = 1.495259e-09
 Solve Converged!

*** Warning ***
/home/wk9874/Documents/simvue/integrations/workspace/copper_mug.i:131.5:
The following warning occurred in the UserObject 'handle-too-hot' of type Terminator.

Terminator 'handle-too-hot' is causing the execution to terminate.

Finished Executing 

And tracked values like this:

        self.file_monitor.tail(
            path_glob_exprs=str(
                pathlib.Path(self._output_dir_path).joinpath(
                    f"{self._results_prefix}.txt"
                )
            ),
            callback=self._per_event_callback,
            tracked_values=[
                re.compile(r"Time Step.*"),
                " Solve Converged!",
                " Solve Did NOT Converge!",
                "Finished Executing",
                re.compile(r"Terminator '.+' is causing the execution to terminate."),
                re.compile(r" \d+ Nonlinear \|R\|"),
                re.compile(r"     \d+ Linear \|R\|"),
            ],

However when I set this file to write line by line (each write separated by 0.01s), then flushing the final line before waiting for 2 seconds and then closing the file, I sometimes get this out of the print statement:

{'nonlinear': ' 2 Nonlinear |R|'}
{'converged': ' Solve Converged!'}
{}
{}
{}
{}
{}
{'terminated': "Terminator 'handle-too-hot' is causing the execution to terminate."}

Note that it is missing the final line.

Although this doesn't always happen... leads me to believe that it is some kind of timing issue or race condition?

Environment (please complete the following information):

  • OS: Ubuntu 22.04
  • Python version: 3.10

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions