Skip to content

autumnust/LoggingRepo

Repository files navigation

Use:
1. Combined all the output 'hx_Extra_spark.log' into a file named 'combined'
2. run `Python extra_logParser.py` 
3. Check the result in `output`

Note:
- k_10_iter_1_part_4_10M inside /history_plus_log is for history server to render the DAG, so that we can compare the result in output with the DAG. 
- In the top and middle (around line 50) of `extra_logParser.py` there's explanation of format for the output.

--- 

This repo is storing all the logging to parse the result. 
Inside each of folder, there's configuration  parameter list for that application. 

The ultimate local version is in /Users/Sun/Dropbox/16Fall/Capstone/LoggingRepo/

About

Spark experiment Logging repo

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 2

  •  
  •  

Languages