Model infer lanes visualizaiton messy

[ketherr]

Dear author, hello
When I loaded the model and config in the project, the inferred lanes are messy and irregular
May I ask what is the reason? Thanks

[ketherr]

No results are obtained at the default threshold（0.4）. If the threshold is lowered to 0.1, the output lane lines are messy, as shown below:

[GSCWW]

I have almost the same problem as you, did you solve it?

[ketherr]

I have almost the same problem as you, did you solve it?
I noticed that the author shared the first version of the model in March, and maybe for some reason the model was modified in June, so I suggest that you can train it yourself and try it

[foww-0001]

I Test the mAP with the weights and config, the value is very low (0.09)

[yutong-yang-mb]

I Test the mAP with the weights and config, the value is very low (0.09)

same here, tested on nuscenes dataset mini split: the mAP is around 0.096. Did you solve it? @foww-0001 @ketherr

[Eragon404]

i got the same problem like this in mini，have you ever tried training by yourself？
The model and loaded state dict do not match exactly
unexpected key in source state_dict: pts_bbox_head.transformer.encoder.layers.0.attentions.1.attention.grid_offsets, pts_bbox_head.transformer.encoder.layers.1.attentions.1.attention.grid_offsets, pts_bbox_head.transformer.encoder.layers.2.attentions.1.attention.grid_offsets

{'NuscMap_chamfer/divider_AP': 0.11277661845088005, 'NuscMap_chamfer/ped_crossing_AP': 0.04197031740720073, 'NuscMap_chamfer/boundary_AP': 0.13267623314944407, 'NuscMap_chamfer/mAP': 0.09580772300250828, 'NuscMap_chamfer/divider_AP_thr_0.2': 0.008244968950748444, 'NuscMap_chamfer/divider_AP_thr_0.5': 0.04119854047894478, 'NuscMap_chamfer/divider_AP_thr_1.0': 0.11810879409313202, 'NuscMap_chamfer/divider_AP_thr_1.5': 0.17902252078056335, 'NuscMap_chamfer/ped_crossing_AP_thr_0.2': 0.0, 'NuscMap_chamfer/ped_crossing_AP_thr_0.5': 0.003289473708719015, 'NuscMap_chamfer/ped_crossing_AP_thr_1.0': 0.021606342867016792, 'NuscMap_chamfer/ped_crossing_AP_thr_1.5': 0.1010151356458664, 'NuscMap_chamfer/boundary_AP_thr_0.2': 5.2317127483547665e-06, 'NuscMap_chamfer/boundary_AP_thr_0.5': 0.0019524672534316778, 'NuscMap_chamfer/boundary_AP_thr_1.0': 0.11499270796775818, 'NuscMap_chamfer/boundary_AP_thr_1.5': 0.28108352422714233}

i try with the original code from gitee, which is not covered by any changes,and i found it works,but still have the warning unexpected key in source state_dict; @ketherr @foww-0001 @GSCWW @yutong-yang-mb
{'NuscMap_chamfer/divider_AP': 0.9405746062596639, 'NuscMap_chamfer/ped_crossing_AP': 0.990001916885376, 'NuscMap_chamfer/boundary_AP': 0.7620361844698588, 'NuscMap_chamfer/mAP': 0.8975375692049662, 'NuscMap_chamfer/divider_AP_thr_0.2': 0.5331693291664124, 'NuscMap_chamfer/divider_AP_thr_0.5': 0.8838977217674255, 'NuscMap_chamfer/divider_AP_thr_1.0': 0.9616131782531738, 'NuscMap_chamfer/divider_AP_thr_1.5': 0.9762129187583923, 'NuscMap_chamfer/ped_crossing_AP_thr_0.2': 0.11844392865896225, 'NuscMap_chamfer/ped_crossing_AP_thr_0.5': 0.9776298999786377, 'NuscMap_chamfer/ped_crossing_AP_thr_1.0': 0.9961879253387451, 'NuscMap_chamfer/ped_crossing_AP_thr_1.5': 0.9961879253387451, 'NuscMap_chamfer/boundary_AP_thr_0.2': 0.012591887265443802, 'NuscMap_chamfer/boundary_AP_thr_0.5': 0.5558196306228638, 'NuscMap_chamfer/boundary_AP_thr_1.0': 0.8317874670028687, 'NuscMap_chamfer/boundary_AP_thr_1.5': 0.898501455783844}

[foww-0001]

i got the same problem like this in mini，have you ever tried training by yourself？ The model and loaded state dict do not match exactly unexpected key in source state_dict: pts_bbox_head.transformer.encoder.layers.0.attentions.1.attention.grid_offsets, pts_bbox_head.transformer.encoder.layers.1.attentions.1.attention.grid_offsets, pts_bbox_head.transformer.encoder.layers.2.attentions.1.attention.grid_offsets

{'NuscMap_chamfer/divider_AP': 0.11277661845088005, 'NuscMap_chamfer/ped_crossing_AP': 0.04197031740720073, 'NuscMap_chamfer/boundary_AP': 0.13267623314944407, 'NuscMap_chamfer/mAP': 0.09580772300250828, 'NuscMap_chamfer/divider_AP_thr_0.2': 0.008244968950748444, 'NuscMap_chamfer/divider_AP_thr_0.5': 0.04119854047894478, 'NuscMap_chamfer/divider_AP_thr_1.0': 0.11810879409313202, 'NuscMap_chamfer/divider_AP_thr_1.5': 0.17902252078056335, 'NuscMap_chamfer/ped_crossing_AP_thr_0.2': 0.0, 'NuscMap_chamfer/ped_crossing_AP_thr_0.5': 0.003289473708719015, 'NuscMap_chamfer/ped_crossing_AP_thr_1.0': 0.021606342867016792, 'NuscMap_chamfer/ped_crossing_AP_thr_1.5': 0.1010151356458664, 'NuscMap_chamfer/boundary_AP_thr_0.2': 5.2317127483547665e-06, 'NuscMap_chamfer/boundary_AP_thr_0.5': 0.0019524672534316778, 'NuscMap_chamfer/boundary_AP_thr_1.0': 0.11499270796775818, 'NuscMap_chamfer/boundary_AP_thr_1.5': 0.28108352422714233}

i try with the original code from gitee, which is not covered by any changes,and i found it works,but still have the warning unexpected key in source state_dict; @ketherr @foww-0001 @GSCWW @yutong-yang-mb {'NuscMap_chamfer/divider_AP': 0.9405746062596639, 'NuscMap_chamfer/ped_crossing_AP': 0.990001916885376, 'NuscMap_chamfer/boundary_AP': 0.7620361844698588, 'NuscMap_chamfer/mAP': 0.8975375692049662, 'NuscMap_chamfer/divider_AP_thr_0.2': 0.5331693291664124, 'NuscMap_chamfer/divider_AP_thr_0.5': 0.8838977217674255, 'NuscMap_chamfer/divider_AP_thr_1.0': 0.9616131782531738, 'NuscMap_chamfer/divider_AP_thr_1.5': 0.9762129187583923, 'NuscMap_chamfer/ped_crossing_AP_thr_0.2': 0.11844392865896225, 'NuscMap_chamfer/ped_crossing_AP_thr_0.5': 0.9776298999786377, 'NuscMap_chamfer/ped_crossing_AP_thr_1.0': 0.9961879253387451, 'NuscMap_chamfer/ped_crossing_AP_thr_1.5': 0.9961879253387451, 'NuscMap_chamfer/boundary_AP_thr_0.2': 0.012591887265443802, 'NuscMap_chamfer/boundary_AP_thr_0.5': 0.5558196306228638, 'NuscMap_chamfer/boundary_AP_thr_1.0': 0.8317874670028687, 'NuscMap_chamfer/boundary_AP_thr_1.5': 0.898501455783844}

what is the diff between gitee and gitlab

[Eragon404]

i got the same problem like this in mini，have you ever tried training by yourself？ The model and loaded state dict do not match exactly unexpected key in source state_dict: pts_bbox_head.transformer.encoder.layers.0.attentions.1.attention.grid_offsets, pts_bbox_head.transformer.encoder.layers.1.attentions.1.attention.grid_offsets, pts_bbox_head.transformer.encoder.layers.2.attentions.1.attention.grid_offsets

{'NuscMap_chamfer/divider_AP': 0.11277661845088005, 'NuscMap_chamfer/ped_crossing_AP': 0.04197031740720073, 'NuscMap_chamfer/boundary_AP': 0.13267623314944407, 'NuscMap_chamfer/mAP': 0.09580772300250828, 'NuscMap_chamfer/divider_AP_thr_0.2': 0.008244968950748444, 'NuscMap_chamfer/divider_AP_thr_0.5': 0.04119854047894478, 'NuscMap_chamfer/divider_AP_thr_1.0': 0.11810879409313202, 'NuscMap_chamfer/divider_AP_thr_1.5': 0.17902252078056335, 'NuscMap_chamfer/ped_crossing_AP_thr_0.2': 0.0, 'NuscMap_chamfer/ped_crossing_AP_thr_0.5': 0.003289473708719015, 'NuscMap_chamfer/ped_crossing_AP_thr_1.0': 0.021606342867016792, 'NuscMap_chamfer/ped_crossing_AP_thr_1.5': 0.1010151356458664, 'NuscMap_chamfer/boundary_AP_thr_0.2': 5.2317127483547665e-06, 'NuscMap_chamfer/boundary_AP_thr_0.5': 0.0019524672534316778, 'NuscMap_chamfer/boundary_AP_thr_1.0': 0.11499270796775818, 'NuscMap_chamfer/boundary_AP_thr_1.5': 0.28108352422714233}

i try with the original code from gitee, which is not covered by any changes,and i found it works,but still have the warning unexpected key in source state_dict; @ketherr @foww-0001 @GSCWW @yutong-yang-mb {'NuscMap_chamfer/divider_AP': 0.9405746062596639, 'NuscMap_chamfer/ped_crossing_AP': 0.990001916885376, 'NuscMap_chamfer/boundary_AP': 0.7620361844698588, 'NuscMap_chamfer/mAP': 0.8975375692049662, 'NuscMap_chamfer/divider_AP_thr_0.2': 0.5331693291664124, 'NuscMap_chamfer/divider_AP_thr_0.5': 0.8838977217674255, 'NuscMap_chamfer/divider_AP_thr_1.0': 0.9616131782531738, 'NuscMap_chamfer/divider_AP_thr_1.5': 0.9762129187583923, 'NuscMap_chamfer/ped_crossing_AP_thr_0.2': 0.11844392865896225, 'NuscMap_chamfer/ped_crossing_AP_thr_0.5': 0.9776298999786377, 'NuscMap_chamfer/ped_crossing_AP_thr_1.0': 0.9961879253387451, 'NuscMap_chamfer/ped_crossing_AP_thr_1.5': 0.9961879253387451, 'NuscMap_chamfer/boundary_AP_thr_0.2': 0.012591887265443802, 'NuscMap_chamfer/boundary_AP_thr_0.5': 0.5558196306228638, 'NuscMap_chamfer/boundary_AP_thr_1.0': 0.8317874670028687, 'NuscMap_chamfer/boundary_AP_thr_1.5': 0.898501455783844}

what is the diff between gitee and gitlab

I have no idea, I tend to think that the author made some modifications when adapting the Argoverse 2 dataset, which led to the unavailability of nuscence;The version on gitee has not undergone any refactoring

[fishmarch]

Hi, sorry for the late reply. This may be the reason #19.

[yutong-yang-mb]

Hi Xiaoyu, thanks for your reply! Yes I think this is the issue. Moreover, I also trained a MapQR model for 24 epochs (with the @auto_fp16() commended out), and finally achieved mAP of 0.65723 under the 0.5,1.0 and 1.5 distance critarias. I used the identical config as the repo and didn't change anything.

This mAP is slightly worse than the mAP reported in the paper 0.664. Do you think that the performence drop is because of the fp16 vs. fp32? Thank you in advance! @fishmarch

Below is my eval results:
{"mode": "val", "epoch": 24, "iter": 753, "lr": 0.0, "NuscMap_chamfer/divider_AP": 0.67918, "NuscMap_chamfer/ped_crossing_AP": 0.6198, "NuscMap_chamfer/boundary_AP": 0.67272, "NuscMap_chamfer/mAP": 0.65723, "NuscMap_chamfer/divider_AP_thr_0.2": 0.19781, "NuscMap_chamfer/divider_AP_thr_0.5": 0.55179, "NuscMap_chamfer/divider_AP_thr_1.0": 0.7107, "NuscMap_chamfer/divider_AP_thr_1.5": 0.77504, "NuscMap_chamfer/ped_crossing_AP_thr_0.2": 0.04969, "NuscMap_chamfer/ped_crossing_AP_thr_0.5": 0.40628, "NuscMap_chamfer/ped_crossing_AP_thr_1.0": 0.67594, "NuscMap_chamfer/ped_crossing_AP_thr_1.5": 0.77717, "NuscMap_chamfer/boundary_AP_thr_0.2": 0.03219, "NuscMap_chamfer/boundary_AP_thr_0.5": 0.45054, "NuscMap_chamfer/boundary_AP_thr_1.0": 0.74342, "NuscMap_chamfer/boundary_AP_thr_1.5": 0.82419}

[fishmarch]

Hi Xiaoyu, thanks for your reply! Yes I think this is the issue. Moreover, I also trained a MapQR model for 24 epochs (with the @auto_fp16() commended out), and finally achieved mAP of 0.65723 under the 0.5,1.0 and 1.5 distance critarias. I used the identical config as the repo and didn't change anything.

This mAP is slightly worse than the mAP reported in the paper 0.664. Do you think that the performence drop is because of the fp16 vs. fp32? Thank you in advance! @fishmarch

Below is my eval results: {"mode": "val", "epoch": 24, "iter": 753, "lr": 0.0, "NuscMap_chamfer/divider_AP": 0.67918, "NuscMap_chamfer/ped_crossing_AP": 0.6198, "NuscMap_chamfer/boundary_AP": 0.67272, "NuscMap_chamfer/mAP": 0.65723, "NuscMap_chamfer/divider_AP_thr_0.2": 0.19781, "NuscMap_chamfer/divider_AP_thr_0.5": 0.55179, "NuscMap_chamfer/divider_AP_thr_1.0": 0.7107, "NuscMap_chamfer/divider_AP_thr_1.5": 0.77504, "NuscMap_chamfer/ped_crossing_AP_thr_0.2": 0.04969, "NuscMap_chamfer/ped_crossing_AP_thr_0.5": 0.40628, "NuscMap_chamfer/ped_crossing_AP_thr_1.0": 0.67594, "NuscMap_chamfer/ped_crossing_AP_thr_1.5": 0.77717, "NuscMap_chamfer/boundary_AP_thr_0.2": 0.03219, "NuscMap_chamfer/boundary_AP_thr_0.5": 0.45054, "NuscMap_chamfer/boundary_AP_thr_1.0": 0.74342, "NuscMap_chamfer/boundary_AP_thr_1.5": 0.82419}

Hi, this may be one of the reasons. But, I typically got results around 66.0, and there's some inherent randomness