Runtime independent design for Rust APIs#17
Runtime independent design for Rust APIs#17qor-lb merged 5 commits intoeclipse-score:main_api_rsfrom
Conversation
* Primary APIs moved to Runtime traits * Add generic compatibilty for Vehicle Consumer and Producer * Added Provider and Consumer InstanceInfo
|
|
||
| fn main() { | ||
| let runtime_builder = RuntimeBuilderImpl::new(); | ||
| let runtime = Builder::<MockRuntimeImpl>::build(runtime_builder).unwrap(); |
There was a problem hiding this comment.
I expected the non mocked runtime is used here
There was a problem hiding this comment.
Yes , both can be used.
Added in next commit.
com-api/com-api-concept/src/lib.rs
Outdated
| /// Reading from the received pointer before initialization is undefined behavior. | ||
| fn as_mut_ptr(&mut self) -> *mut T; |
There was a problem hiding this comment.
should it be unsafe used if undefined behavior can happen?
There was a problem hiding this comment.
No, we don't, as the method itself will not introduce UB - using the pointer could, though. However, any operation that needs dereferencing is anyway unsafe on the pointer itself, so that's already how it should be.
com-api/com-api-concept/src/lib.rs
Outdated
| { | ||
| } | ||
|
|
||
| pub trait Subscriber<T: Reloc + Send, R: Runtime + ?Sized,> { |
There was a problem hiding this comment.
I am surprised that generated code for both the mock and the lola binding exist. At the moment it looks even identical.
I was expecting that if we use generics the same generated code can be used for both bindings.
There was a problem hiding this comment.
Yes. @bharatGoswami8 can we remove both lola and mock generated crates and create top level crate com-api-gen-types ? If you need smoene doing this, let me know and we can add commits.
There was a problem hiding this comment.
Yes, now we have only one gen file with generic R
pawelrutkaq
left a comment
There was a problem hiding this comment.
We are heading to implement iceoryx2 and see if we have any issues with other backends.
| type Publisher<T: Reloc + Send>: Publisher<T>; | ||
| type ProviderInfo: Send + Clone; | ||
| type ConsumerInfo: Send + Clone; | ||
|
|
There was a problem hiding this comment.
General note. Can we suffix here the types with Type suffix or any other name that is different than the bounded Trait So we don't have type Publisher: Publisher. This is really confusing later in the code and easy to miss.
There was a problem hiding this comment.
As of now we don't have any guideline for naming in s-core coding guideline .... https://eclipse-score.github.io/score/main/contribute/development/rust/coding_guidelines.html
we may need to get the naming guideline from s-core rust community.
There was a problem hiding this comment.
Can we at least don't name different things with exactly same name?
As Paweł pointed we have type Publisher with assigned struct Publisher that implements trait Publisher.
I don't think we need explicit rule for that.
| &self, | ||
| _instance_specifier: InstanceSpecifier, | ||
| ) -> Self::ServiceDiscovery<I>; | ||
|
|
There was a problem hiding this comment.
I guess async api we can add in next PR.
com-api/com-api-concept/src/lib.rs
Outdated
| fn check_str(_service_name: &str) -> bool { | ||
| // For now, accept any non-empty string as a valid service name | ||
| // In a real implementation, this might validate the format | ||
| true |
There was a problem hiding this comment.
This means that we will have to specify the str layout in S-CORE as it has to be runtime agnostic.
There was a problem hiding this comment.
Either we can call the backend compatible APIs or we can have validation in com-api.
I think we should use backend API so that if communication medium or channel change we no need to modify validation logic in com-api side.
There was a problem hiding this comment.
Thats a point ofc. But this would mean that if you change a backend (runtime) then You must change InstanceSpecifier across code base. This would contradict the whole idea of current work.
There was a problem hiding this comment.
InstanceSpecifier is definitely meant to be backend agnostic. All backend specific stuff has to be buried inside ProducerInfo or ConsumerInfo.
There was a problem hiding this comment.
Then I guess here we will just do like allowed letters , numbers and / ie and thats it.
There was a problem hiding this comment.
Decision: Enforce format in concept level (backend agnostic) as described in doc of InstanceSpecifier
com-api/com-api-concept/src/lib.rs
Outdated
| /// The string shall describe where to find a certain instance of a service. Each level shall look | ||
| /// like this | ||
| /// <InterfaceName>:my/path/to/service_name | ||
| #[derive(Clone)] |
There was a problem hiding this comment.
missing Debug, PartialEq, PartialOrd, Ord,Eq
|
|
||
| unsafe impl Reloc for () {} | ||
| unsafe impl Reloc for u32 {} | ||
|
|
There was a problem hiding this comment.
We shall add reloc for all u*, all i*, float*, bool,
There was a problem hiding this comment.
Missing:
usize
u128
i128
isize
f64
all atomics ?
[T] where T: Reloc
[T; N] where T: Reloc
core::mem::MaybeUninit where T: Reloc
tuples from T1, to T5 ie (so variants from 1 elem to 5 elems) where TX: Reloc
There was a problem hiding this comment.
I would move reloc trait with this to separate file reloc.rs since we will also need to add the Reloc derive proc macro and reexport it from there.
| sample.send().unwrap(); | ||
| } | ||
|
|
||
| fn main() { |
There was a problem hiding this comment.
we shall have the main running the same code for MockRuntime and LolaRuntime. So we can see all that works together.
| @@ -67,56 +74,19 @@ mod test { | |||
| fn create_producer() { | |||
There was a problem hiding this comment.
We need to add an example with composition.
pub struct SomeClass {
// Keeps VehicleConsumer, or its part
}
So we can write UT that shows the SomeClass works with SomeRuntime and MockRuntime and is testable due to it.
There was a problem hiding this comment.
Added VehicleMonitor , just added for Consumer.
Will extend this with producer also.
| use std::fmt::Debug; | ||
| use std::future::Future; | ||
| use std::ops::{Deref, DerefMut}; | ||
| use std::path::Path; |
There was a problem hiding this comment.
change whats possible to core instead std.
[workspace.lints.clippy]
std_instead_of_core = "warn"
alloc_instead_of_core = "warn"
and enable warnings as errors or use err here.
| type Subscriber<T: Reloc + Send>: Subscriber<T, Self>; | ||
| type ProducerBuilder<I: Interface, P: Producer<Self, Interface = I>>: ProducerBuilder<I, P, Self>; | ||
| type Publisher<T: Reloc + Send>: Publisher<T>; | ||
| type ProviderInfo: Send + Clone; |
There was a problem hiding this comment.
Yes ProviderInfo added as instance info of Producer , I think this will be use with offer service method but need to check feasibility.
This is added just to notify we have two different type for Producer and Consumer.
com-api/com-api-concept/src/lib.rs
Outdated
There was a problem hiding this comment.
Should we make all samples implement Debug by requiring it for T ? later is's hard to debug in code without it.
|
Working on the review comments... |
* Added Async API for get_instance * added Debug trait for All the Sample trait * Added Core instead of std all the possible palce * Added one Gen file and remove runtime specific gen file * Added both Mock and Lola time in same main file * Added VehicleMonitor as Consumer on example * Removed all the feature flag * added impl for Reloc * Removed root toml file
piotrchodorowski
left a comment
There was a problem hiding this comment.
I implemented Iceoryx integration on top of those changes and added comments with all my findings.
I didn't add any comments in runtime part as those changes should results from proposed changes in concept part.
com-api/com-api-concept/src/lib.rs
Outdated
| #[derive(Clone, Debug)] | ||
| pub struct InstanceSpecifier { | ||
| pub specifier: String, | ||
| specifier: Option<String>, |
There was a problem hiding this comment.
Why removed pub? I think we need it
There was a problem hiding this comment.
Why? If you want to access the string, you can just as well use AsRef<str>. We can also add Into<Option<String>> if that's what would fit your purpose.
There was a problem hiding this comment.
But without pub I cant access this field within runtime e.g:
|
280 | let specifier = instance_info.instance_specifier.specifier.clone().unwrap();
| ^^^^^^^^^ private field
There was a problem hiding this comment.
We can access using instanceSpecifier instance rather then making specifier field as public.
We have new() which is returning InstanceSpecifier.
There was a problem hiding this comment.
Okay, it is clear to me now how to use it
com-api/com-api-concept/src/lib.rs
Outdated
| type Interface: Interface; | ||
| type OfferedProducer: OfferedProducer<Interface = Self::Interface>; | ||
| type OfferedProducer: OfferedProducer<R, Interface = Self::Interface>; | ||
|
|
There was a problem hiding this comment.
We will need new here
| fn new(instance_info: R::ProviderInfo) -> Self; |
There was a problem hiding this comment.
The chain here should be Runtime::producer_builder().build(). Would that fit as well?
There was a problem hiding this comment.
Yes, it was needed to be able to crate Producer with instance_info from ProducerBuilder.
| type SampleMaybeUninit<'a>: SampleMaybeUninit<T> + 'a | ||
| where | ||
| Self: 'a; | ||
|
|
There was a problem hiding this comment.
We will need new here
| fn new(identifier: &str, instance_info: R::ProviderInfo) -> Self; |
There was a problem hiding this comment.
The chain here should be Runtime::producer_builder().build().offer().
There was a problem hiding this comment.
Yes, it is needed to be able to crate Publisher with identifier and instance info passed.
In current implementation there is no way to provide service name for Publisher.
|
|
||
| pub struct VehicleProducer<R: Runtime + ?Sized> | ||
| { | ||
| _runtime: core::marker::PhantomData<R>, |
There was a problem hiding this comment.
| _runtime: core::marker::PhantomData<R>, | |
| _runtime: core::marker::PhantomData<R>, | |
| instance_info: R::ProviderInfo, |
| impl<R: Runtime + ?Sized> Producer<R> for VehicleProducer<R> { | ||
| type Interface = VehicleInterface; | ||
| type OfferedProducer = VehicleOfferedProducer<R>; | ||
|
|
There was a problem hiding this comment.
| fn new(instance_info: R::ProviderInfo) -> Self { | |
| VehicleProducer { | |
| _runtime: std::marker::PhantomData, | |
| instance_info, | |
| } | |
| } |
README.md
Outdated
| For mock test: | ||
| ``` | ||
| inc_mw_com/com-api$ cargo run --example basic-consumer-producer --features "lola" | ||
| inc_mw_com$ cargo test --test basic-consumer-producer-test |
There was a problem hiding this comment.
This should be called within com-api folder
There was a problem hiding this comment.
will update on latest commit.
README.md
Outdated
| ``` | ||
|
|
||
| For LoLa build: | ||
| For mock test: |
There was a problem hiding this comment.
will update on latest commit.
README.md
Outdated
| inc_mw_com$ cargo test --test basic-consumer-producer-test | ||
| ``` | ||
|
|
||
| For mock Build and Run: |
There was a problem hiding this comment.
will update on latest commit.
| pub fn monitor_tire_data(self) -> Result<String> { | ||
| let subscribed = self.consumer.left_tire.subscribe(3)?; | ||
| let mut sample_buf = SampleContainer::new(); | ||
|
|
There was a problem hiding this comment.
Why removed logic with sending and receiving n samples?
| sample.send().unwrap(); | ||
| } | ||
|
|
||
| fn run_with_runtime<R: Runtime>(name: &str, runtime: &R) { |
There was a problem hiding this comment.
I recommend to change this example logic to something like:
let offered_producer = create_offered_producer(&runtime);
let consumer = create_consumer(&runtime);
let monitor = VehicleMonitor::new(offered_producer, consumer);
for _ in 0..SAMPLES_NUM {
monitor.produce_tire_data().unwrap();
monitor.consume_tire_data().unwrap();
}
|
@darkwisebear I made a check on trait issues we discussed and doing |
|
@bharatGoswami8 One small note: after some additional investigation I see that it would be good (or even it could be necessary) to make all new() calls returning Result instead of just Self. Including of course the two new()'s proposed by myself before. |
As discussed in this comment thread #17 (comment) , we will update new method with returning Result. |
|
Working on pending review comments fix, I will share latest commit next week start... |
Sounds ok. We could add a feature flag so that the derive macro will generate the impl, guarded by the optional dependency on the crate that offers |
* Added ProviderInfo instance in producer struct and API side * Added Example in detail * Subscriber API pass self as refrence * Added eraly fail check for consumer and producer * Added Reloc in seprate file and used as module * Added Type id with interface trait * Added AsMut trait bound with SampleMaybeUninit trait * Validation for Service name * InstanceSpecifier enum added with ANY and specific instance name * Find Service API take FindServiceSpecifier enum as argument * instance id removed and instead of that used instance specifier to identify the insatnce
com-api/com-api-concept/src/lib.rs
Outdated
| fn find_service<I: Interface>( | ||
| &self, | ||
| _instance_specifier: InstanceSpecifier, | ||
| _instance_specifier: FindServiceSpecifier, |
There was a problem hiding this comment.
Yes, will remove in next commit.
| && service_name.split('/').all(|parts| { | ||
| !parts.is_empty() && parts.bytes().all(|part| part.is_ascii_alphanumeric()) | ||
| }) | ||
| } |
There was a problem hiding this comment.
Validation does not match format from doc <InterfaceName>:my/path/to/service_name. Since we put InterfaceName into TYPE_ID of Interface, I guess doc shall be my/path/to/service_name . I would suggest to start also with leading /
There was a problem hiding this comment.
updated the document and start path with leading slash.
com-api/com-api-concept/src/lib.rs
Outdated
| @@ -152,6 +154,11 @@ impl AsRef<str> for InstanceSpecifier { | |||
| .unwrap_or("[ANY]") | |||
There was a problem hiding this comment.
InstanceSpecifier now have no Any. So the self.specifier cannot be optional anymore and there is no [ANY] case
There was a problem hiding this comment.
Ok in that case we need to return default " " string if error -> .unwrap_or("")
There was a problem hiding this comment.
Just remove the Option in the type definition. It's not needed anymore. An oversight on my side in the internal review.
| Specific(InstanceSpecifier), | ||
| Any, | ||
| } | ||
|
|
There was a problem hiding this comment.
We shall add Into<FindServiceSpecifier> for InstanceSpecifier.
There was a problem hiding this comment.
Yes this will add in next commit.
com-api/com-api-concept/src/lib.rs
Outdated
| unsafe impl Reloc for () {} | ||
| unsafe impl Reloc for u32 {} | ||
| // Reloc trait and its implementations have been moved to reloc.rs | ||
| // Use `use reloc::Reloc;` to import it here and in other files. |
There was a problem hiding this comment.
Yes , removed it was added to notify the reviewer.
com-api/com-api-concept/src/lib.rs
Outdated
|
|
||
| pub trait ConsumerDescriptor<R: Runtime + ?Sized> { | ||
| fn get_instance_id(&self) -> usize; // TODO: Turn return type into separate type | ||
| fn get_instance_identifier(&self) -> String; |
There was a problem hiding this comment.
This shall return &InstanceSpecifier and not String
|
|
||
| unsafe impl Reloc for () {} | ||
| unsafe impl Reloc for u32 {} | ||
| unsafe impl Reloc for u64 {} |
com-api/com-api-concept/Cargo.toml
Outdated
| publish = ["common"] | ||
| license = "Apache-2.0" | ||
|
|
||
| [dependencies] |
There was a problem hiding this comment.
Updated on latest commit.
| [dependencies] | ||
| com-api-concept = { workspace = true } | ||
|
|
||
|
|
There was a problem hiding this comment.
Updated on latest commit.
* Added Reloc u8,u16 and i8 ,i16 * Removed comment
| use_producer(runtime); | ||
| use_consumer(runtime); | ||
|
|
||
| let monitor = VehicleMonitor::new(use_consumer(runtime), use_producer(runtime)); |
There was a problem hiding this comment.
| let monitor = VehicleMonitor::new(use_consumer(runtime), use_producer(runtime)); | |
| let producer = use_producer(runtime); | |
| let consumer = use_consumer(runtime); | |
| let monitor = VehicleMonitor::new(consumer, producer); |
How about making producer first to make sure consumer will find suitable one?
There was a problem hiding this comment.
yes in this example we can move it., but both should be independent of each other.
There was a problem hiding this comment.
Thats not correct I think.
We can't just create Consumer, we can only create Consumer for previously discovered Producer so correct order of operation is mandatory.
| type Subscription = SubscriberImpl<T>; | ||
| fn new(identifier: &str, instance_info: LolaConsumerInfo) -> Self { | ||
| Self { | ||
| fn new(identifier: &str, instance_info: LolaConsumerInfo) -> com_api_concept::Result<Self> { |
There was a problem hiding this comment.
pub fn new() is no longer needed
There was a problem hiding this comment.
new is not pub it is from SubscribableImpl trait.
| where | ||
| Self: 'a; | ||
|
|
||
| fn new(_identifier: &str, _instance_info: LolaProviderInfo) -> com_api_concept::Result<Self> { |
There was a problem hiding this comment.
pub fn new() is no longer needed
There was a problem hiding this comment.
it is not public , it is from Publisher trait.
There was a problem hiding this comment.
Sorry I put that comment (and the second one in SubscribableImpl) in misleading place.
pub fn new() is no longer needed since we have fn new(_identifier: &str, _instance_info: LolaProviderInfo)
Same for SubscribableImpl.
| // Example struct demonstrating composition with VehicleConsumer | ||
| pub struct VehicleMonitor<R: Runtime> { | ||
| consumer: VehicleConsumer<R>, | ||
| producer: VehicleOfferedProducer<R>, |
There was a problem hiding this comment.
| producer: VehicleOfferedProducer<R>, | |
| producer: VehicleOfferedProducer<R>, | |
| tire_subscriber: <<R as Runtime>::Subscriber<Tire> as Subscriber<Tire, R>>::Subscription, |
Subscriber should be created before sending data to make sure those wont be dropped
| pub fn new(consumer: VehicleConsumer<R>) -> Self { | ||
| Self { consumer } | ||
| pub fn new(consumer: VehicleConsumer<R>, producer: VehicleOfferedProducer<R>) -> Self { | ||
| Self { consumer, producer } |
There was a problem hiding this comment.
| Self { consumer, producer } | |
| let tire_subscriber = consumer.left_tire.subscribe(3).unwrap(); | |
| Self { consumer, producer, tire_subscriber } |
Subscriber should be created before sending data to make sure those wont be dropped
* Updated check_str validation logic * Updated example file * Removed unused new method * Added Into Trait impl for InstanceSpecifier
f982cd5 to
4dd0b65
Compare
Uh oh!
There was an error while loading. Please reload this page.