Skip to content

Conversation

@hofi1
Copy link
Owner

@hofi1 hofi1 commented Sep 19, 2023

Problem:

The string in the assertion can change, because the Map which is used to store the data in the JSONObject returns the data in non-deterministic order.
The flaky test was found by using the NonDex tool.

Solution:

Change the assertion is to a JSONAssertion (e.g. the order of child elements in an element do not matter)

Result:

The test is deterministic and not flaky. This improves the quality of the test and reduces the time to search for the bug during future development.

Reproduce:
mvn edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.json.junit.JSONMLTest#testToJSONObject_reversibility

@Einsteinnnnn
Copy link

Hi,
Your fix looks good to me, but I have some suggestions about your PR.
I learned from other students that to provide a clearer context for reviewers and future reference, it would be beneficial to edit the pull request description with specific headings such as:

Cause: Describe the underlying issue or bug that this pull request aims to solve.
Steps to Reproduce: Provide a detailed set of steps that demonstrate how to reproduce the problem. This will help in understanding the issue thoroughly.
Proposed Solution: Explain the changes you've made and how they address the problem identified in the "Cause" section.

And include some code snaps in your PR description can make it easier for others to know the root cause of the flaky test and how to fix it.

@hofi1 hofi1 changed the title fix: flakiness in JSONMLTest#testToJSONObject_reversibility fix: flakiness in org.json.junit.JSONMLTest#testToJSONObject_reversibility Sep 29, 2023
@hofi1
Copy link
Owner Author

hofi1 commented Sep 29, 2023

hey @Einsteinnnnn , I just updated the format of my Pull Request, thanks for the feedback, I appreciate it!

@zzjas
Copy link

zzjas commented Oct 7, 2023

Is this test close to the test fixed here? #2

You can consider fix them in one PR so that the added dependency looks more worth it.

Also you used some other dependency here #3

Maybe you want to open one real PR first to kinda see if the developers are okay with the dependency or not.

But anyway, it's all your call. Feel free to do anything. Once you open a real PR, please mark this tentative PR as Opened in your tentative_pr.csv file and also raise a PR to IDoFT marking this as Opened. Thanks!

@hofi1
Copy link
Owner Author

hofi1 commented Oct 12, 2023

@zzjas I merged the two json branches, and opened a real a pull request for the two JSON-requiring fixes.
These two will be deleted as soon as they get merged (#2 and #1 )

I don't think that It is possible to use a different dependency in #3 , but I will check it again

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants