Skip to content

Comments

fix: item db duplicate handling is stable over insertion order#272

Open
BrknRobot wants to merge 3 commits intoamtep:mainfrom
BrknRobot:dbentry_determanism
Open

fix: item db duplicate handling is stable over insertion order#272
BrknRobot wants to merge 3 commits intoamtep:mainfrom
BrknRobot:dbentry_determanism

Conversation

@BrknRobot
Copy link
Contributor

Previously, the particular duplicate stored in the item db depended on insertion order.
Now, item Locs are compared during db insertion to ensure the stored entry is the last one in the load order.

The resulting reports should be stable in cases of a single duplicate. More than that may report duplicate differences in which location overwrote which, but always result in the correct final item overwriting all others

@BrknRobot BrknRobot force-pushed the dbentry_determanism branch from 241e33b to 435c913 Compare August 11, 2025 22:28
@BrknRobot BrknRobot force-pushed the dbentry_determanism branch from 435c913 to f2fed22 Compare August 11, 2025 22:52
@amtep
Copy link
Owner

amtep commented Aug 11, 2025

The insertion order is carefully controlled to override items in the right order. Do you have an example of where it's wrong? Because that needs to be looked at.

@BrknRobot
Copy link
Contributor Author

I'm just noticing now that all the examples I have are tutorial steps. Not sure if that's due to an issue there specifically or if it's more broad, but vanilla doesn't have duplicates across files in other spots

Run 1

warning(duplicate-item): tutorial lesson step is redefined by another tutorial lesson step
    --> [CK3] common/tutorial_lessons/00_tutorial_lessons_reactive.txt
3557 |     lesson_war_final_notes_invalidated_step_1 = {
     |     ^ <-- from here
    --> [CK3] common/tutorial_lessons/00_tutorial_lessons_war.txt
 635 |     lesson_war_final_notes_invalidated_step_1 = {
     |     ^ <-- from here

Run 2

warning(duplicate-item): tutorial lesson step is redefined by another tutorial lesson step
    --> [CK3] common/tutorial_lessons/00_tutorial_lessons_war.txt
 635 |     lesson_war_final_notes_invalidated_step_1 = {
     |     ^ <-- from here
    --> [CK3] common/tutorial_lessons/00_tutorial_lessons_reactive.txt
3557 |     lesson_war_final_notes_invalidated_step_1 = {
     |     ^ <-- from here

Run 1

warning(exact-duplicate-item): tutorial lesson step is redefined by an identical tutorial lesson step
    --> [CK3] common/tutorial_lessons/00_tutorial_lessons_reactive.txt
3355 |     lesson_warscore_step_1 = {
     |     ^ <-- from here
    --> [CK3] common/tutorial_lessons/00_tutorial_lessons_war.txt
 400 |     lesson_warscore_step_1 = {
     |     ^ <-- from here

Run 2

warning(exact-duplicate-item): tutorial lesson step is redefined by an identical tutorial lesson step
    --> [CK3] common/tutorial_lessons/00_tutorial_lessons_war.txt
 400 |     lesson_warscore_step_1 = {
     |     ^ <-- from here
    --> [CK3] common/tutorial_lessons/00_tutorial_lessons_reactive.txt
3355 |     lesson_warscore_step_1 = {
     |     ^ <-- from here

@BrknRobot
Copy link
Contributor Author

vic3 coat of arms and ck3 accessory variations have cross file duplicates but don't seem to be affected actually. I see coa has some custom logic for duplicates, but I'm not seeing why accessory variations are unaffected

@BrknRobot
Copy link
Contributor Author

Possibly the difference is that tutorial steps get added as a subitem

@BrknRobot
Copy link
Contributor Author

It seems that processing subitems in loc order stabilises the tutorial step duplicate reports
BrknRobot@bdfdd48

@amtep
Copy link
Owner

amtep commented Aug 12, 2025

It seems that processing subitems in loc order stabilises the tutorial step duplicate reports BrknRobot@bdfdd48

Okay :)
I'd like to have both patches. Inserting in the correct order, but also making add forgiving of wrong order inserts. The latter is a step toward maybe parallelizing insertions.

But what do you think of doing the same for other users of dup_error? Every FileHandler has one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants