model-conductor

"Conductor" of model construction (frontend api of MDDO demo system).

Directories

+ model-conductor/     # https://github.com/ool-mddo/model-conductor (THIS repository)
  + doc/               # class documents (generated w/yard)
  + lib/               # library for appliation

Setup

Requirements

• Ruby >3.1.0 (development under ruby/3.1.0 and bundler/2.3.5)

Optional: Install ruby gems

# If you install gems into project local
# bundle config set --local path 'vendor/bundle'
bundle install

Environment variables

API entrypoints:

• BATFISH_WRAPPER_HOST : batfish-wrapper host
• NETOMOX_EXP_HOST : netomox-exp host

Log level variable:

• MODEL_CONDUCTOR_LOG_LEVEL (default info)
• select a value from fatal, error, warn, info and debug

Run REST API server

bundle exec rackup -s webrick -o 0.0.0.0 -p 9292

For development: rerun watches file update and reload the server.

• --force-polling in container with volume mount

rerun [--force-polling] bundle exec rackup -s webrick -o 0.0.0.0 -p 9292

REST API

Operate model data

Generate snapshot topology from query data for all snapshots in a network

• POST /conduct/<network>/<snnapshot>/topology
  • label: Label (description) of the physical snapshot
  • phy_ss_only: [optional] Flag to target only physical snapshots
  • use_parallel: [optional] Flag to use parallel processing for query-to-topology data generation stage
  • off_node: [optional] Node name to draw-off
  • off_intf_re: [optional] Interface name (regexp match) to draw-off in off_node

# model-info.json
# -> { "label": "description of the snapshot", ... }
curl -X POST -H "Content-Type: application/json" -d @model-info.json \
  http://localhost:9292/conduct/pushed_configs/mddo_network

Static verification of network structure

Fetch subsets of a snapshot

• GET /conduct/<network>/<snapshot>/subsets

curl http://localhost:9292/conduct/pusheed_configs/mddo_network/subsets

Fetch subset comparison data between physical and logical snapshots in a network

• GET /conduct/<network>/<physical-snapshot>/subsets_diff
  • min_score: [optional] Ignore comparison data lower than this score (default: 0)

curl http://localhost:9292/conduct/pushed_configs/mddo_network/subsets_diff

L3 Reachability test

Run reachability test with test-pattern definition

• GET /conduct/<network>/reachability
  • snapshoots: List of snapshot to test reachability
  • test_pattern: Test pattern definition

# test_pattern.json
# -> { "snapshots": ["mddo_network_linkdown_01", ...], "test_pattern": <test-pattern> }
curl -X POST -H "Content-Type: application/json" -d @test_pattern.json \
  http://localhost:9292/conduct/pushed_configs/reachability

Snapshot diff

Take snapshot diff in a network

• GET /conduct/<network>/snapshot_diff/<src_snapshot>/<dst_snapshot>
  • upper_layer3: [optional] compare layer3 or upper layer

curl -s "http://localhost:9292/conduct/mddo-ospf/snapshot_diff/emulated_asis/emulated_tobe?upper_layer3=true"

Take snapshot diff and write back (overwrite) as destination snapshot

• POST /conduct/<network>/snapshot_diff/<src_snapshot>/<dst_snapshot>
  • upper_layer3: [optional] compare layer3 or upper layer

curl -s -X POST -H 'Content-Type: application/json' \
  -d '{ "upper_layer3": true }' \
  http://localhost:9292/conduct/mddo-ospf/snapshot_diff/emulated_asis/emulated_tobe

Convert snapshot namespace

Convert namespace of source snapshot and post it as destination snapshot

• POST /conduct/<network>/ns_convert/<src_snapshot>/<dst_snapshot>
  • table_origin: [optional] Origin snapshot name to initialize convert table, Force update the convert table of a network if this option used.

curl -s -X POST -H 'Content-Type: application/json' \
  -d '{ "table_origin": "original_asis" }' \
  http://localhost:9292/conduct/mddo-ospf/ns_convert/original_asis/emulated_asis

Splice external/preallocated topology

Splice external topology (external-AS and/or Layer3 pre-allocated resources topology) to snapshot topology.

• POST /conduct/<network>/<snapshot>/splice_topology
  • ext_topology_data: [optional] external topology data (RFC8345 json) to splice
  • l3_preallocated_resources: [optional] preallocated l3 resources (RFC8345 json) to splice'
  • overwrite: [optional] true to write snapshot topology (default: true). If false, it does not modify snapshot topology (Only get spliced topology data)

# ext_topology.json : external topology data to splice (RFC8345 json)
curl -s -X POST -H "Content-Type: application/json" \
  -d @<(jq -s '{ "l3_preallocated_resources": .[0], "ext_topology_data": .[1] }' l3p_topo.json ext_as_topo.json) \
  http://localhost:9292/conduct/mddo-bgp/original_asis/splice_topology

Step-by-step topology operation

For manual-steps usecase.

Generate topology data based on the specified command. Target is original/preallocated(N) snapshot, and it generates next preallocated(N+1) snapshot.

command:

• connect_link: "shut to no-shut" operation
• shutdown_intf: "no-shut to shut" operation

• POST /conduct/<network>/topology_ops
  • command: command name
  • dry_run: [optional] dry-run flag (default:false)
  • args: arguments for each command

curl -s -X POST -H "Content-Type: application/json" \
  -d '{ "command": "connect_link", "args": { "link": { "source": { "node": "as65550-edge01", "tp": "Ethernet3" }, "destination": { "node": "edge-tk12", "tp": "ge-0/0/0.0" } } } } }}' \
  http://localhost:9292/conduct/mddo-bgp/topology_ops

curl -s -X POST -H "Content-Type: application/json" \
  -d '{ "command": "shutdown_intf", "args": { "interface": { "node": "as65550-edge01", "tp": "Ethernet3" } } }' \
  http://localhost:9292/conduct/mddo-bgp/topology_ops

Get current/next target snapshot name of topology-ops API.

• GET /conduct/<network>/topology_ops_targets

curl http://localhost:9292/conduct/mddo-bgp/topology_ops_targets

Add node/term-point attribute data

Note

Currently, for bgp-proc only (to "patch" bgp policy or other attribute data). It used in bgp-policy-parser in PNI use-case of copy-to-emulated-env demo.

Add node/term-point attribute.

• POST /conduct/<network>/<snapshot>/topology/<layer>/policies
  • node: node/term-point attribute (RFC8345-json format)

Patch data (bgp-policy-patch.json)

{
    "node": [
        {
            "node-id": "192.168.255.7",
            "ietf-network-topology:termination-point": [
                {
                    "tp-id": "peer_192.168.255.2",
                    "mddo-topology:bgp-proc-termination-point-attributes": {
                        "import-policy": ["ibgp-export"]
                    }
                }
            ]
        }
    ]
}

curl -s -X POST -H 'Content-Type: application/json' \
  -d @bgp-policy-patch.json \
  http://localhost:9292/conduct/biglobe_deform/original_asis/topology/bgp_proc/policies

Generate candidate config

Generate candidate configs from original_asis snapshot

• POST /conduct/<network>/<snapshot>/candidate_topology
  • phase_number: Number of search iteration (phase)
  • candidate_number: Number of candidate configs
  • usecase: Usecase parameter
    • name: Usecase name
    • sources: Data sources for the usecase

curl -s -X POST -H 'Content-Type: application/json' \
  -d '{"candidate_number": 3, "usecase": { "name": "pni_te", "sources": ["params", "phase_candidate_opts"]}}' \
  http://localhost:9292/conduct/mddo-bgp/original_asis/candidate_topology

NOTE: "phase_candidate_opts" for multi_region_te or multi_src_as_te usecase

"phase_candidate_opts.yaml" can include flow_data for candidate model generation. If with flow_data, candidate model is generated using the flow_data, prefix-set selection done by flow_data. But without flow_data, candidate model is simple selection, prefix-set selection done by sequential.

---
peer_asn: 65550
node: edge-tk01
interface: ge-0/0/1.0
flow_data: flows/event # csv

Development

Optional: Build model-conductor container

model-conductor uses netomox gem that pushed on github packages. So, it need authentication to exec bundle install when building its container image. You have to pass authetication credential via ghp_credential environment variable like below:

• USERNAME : your github username
• TOKEN : your github personal access token (need read:packages scope)

ghp_credential="USERNAME:TOKEN" docker buildx build -t model-conductor --secret id=ghp_credential .

Generate YARD documents

YARD options are in .yardopts file.

bundle exec rake yard

Run yard document server (access http://localhost:8808/ with browser)

bundle exec yard server

Code analysis

bundle exec rake rubocop
# or
bundle exec rake rubocop:auto_correct