[TASK] Deterministic JobId for Service Pallet Blueprints

[shekohex]

Summary

This RFC proposes replacing the current index-based job identification system with a deterministic hash-based JobId system. This change will make jobs self-identifying, simplify deployment, and improve the development experience.

Problem Statement

Currently, the Service Pallet identifies jobs by their index position in a Vec<Job> within a blueprint. This approach has limitations:

1. Jobs are not self-describing of their identity
2. Job IDs are positional rather than content-based
3. Job identity is dependent on deployment order
4. Any change in the Vec order impacts all subsequent job IDs
5. Could introduce potential conflicts if multiple jobs have the same identifier in the blueprint code.

Proposal

We propose implementing a deterministic hashing mechanism for job identification similar to how EVM contracts handle function identification.

Deterministic JobId Generation

For each job, we will generate a deterministic hash as follows:

$$ \begin{aligned} & \text{let } H = \text{Keccak256 Hash function} \\ & \text{let FQN} = \text{Fully Qualified Name of Function} \\ & \text{let jobName} = \text{FQN(job)} \\ & \text{let args} = A \parallel B \parallel C ... \\ & \text{Then} \\ & \text{jobId} = H(\text{jobName} \parallel \text{args})[0..4] \end{aligned} $$

Where:

• H is the Keccak256 hash function
• FQN(job) is the fully qualified name of the function
• args are the concatenated arguments
• We take the first 4 bytes of the hash result

Technical Changes

Storage Changes

1. Change job_id from u8 (1 byte) to u32 (4 bytes)
2. Replace Vec<Job> with BTreeMap<u32, Job> for efficient job lookup

Blueprint SDK Changes

The Blueprint SDK will be updated to:

1. Include methods to compute job IDs deterministically
2. Enable direct job routing using these IDs
3. Provide validation to ensure no job ID collisions within the same blueprint

Benefits

1. Self-describing jobs: Each job computes its own ID based on its function name and parameters
2. Stable IDs: Job IDs won't change when other jobs are added or removed
3. Simplified development: No need to worry about job ordering
4. Better integration with blueprint SDK: Direct routing to jobs
5. More robust system: Reduced chances of errors from job position changes

Potential Challenges

1. Hash collisions: While unlikely with 4 bytes (≈4 billion possibilities) in the same blueprint, we should implement collision detection at build-time of the blueprint itself
2. Backward compatibility: Existing blueprints will need updates, though there is no live production deployment of them yet.
3. Documentation: Clear guidance will be needed on computing and using job IDs

Implementation Plan

1. Create a prototype implementation
2. Add tests for collision detection
3. Update the blueprint SDK to support deterministic job IDs
4. Update the runtime to support the new storage structure
5. Update documentation and examples

Conclusion

Moving to deterministic job IDs will make the Service Pallet more robust, developer-friendly, and flexible. The proposed change aligns with best practices seen in other systems like EVM contracts and will improve the overall developer experience.