[SPIP] fix: DataEntry app not loading due to sessionStorage limits

[MatiasArriola]

Related Issue: https://app.clickup.com/t/8698nqxmt

When loading the DataEntry app as a user with access to a large set of metadata, the app won't load if the size of the metadata response exceeds the 5mb sessionStorage quota allowed by browsers.

fix: use variables instead of sessionStorage to avoid the QuotaExceededError.

At first tried using indexedDb as a replacement, but since I couldn't find references to these sessionStorage keys anywhere else, a simpler approach is to use common in-memory variables.

[tokland]

At first tried using indexedDb as a replacement, but since I couldn't find references to these sessionStorage keys anywhere else, a simpler approach is to use common in-memory variables.

If I understand it correctly, DAO.store provides the service. We'd need to add a new key in objectStores ("metadata", for example):

dhis2-core/dhis-2/dhis-web/dhis-web-dataentry/src/main/webapp/dhis-web-dataentry/javascript/form.js

Line 180 in 44c52fb

objectStores: [ 'optionSets', 'forms' ]

And then we can interact with the object, using that new key. The only expectation is that the object should have the prop "id", which will be used as retrieval key:

> DAO.store.set("metadata", {id: "1234", name: "Mary"})

> DAO.store.get("metadata", "1234").done(console.log); 
Object { name: "Mary", id: "1234" }

In any case, check with @Ramon-Jimenez if we prefer the simple memory cache.

[MatiasArriola]

Thanks for the feedback!

Just to add a little bit of context, I considered it was a good tradeoff to keep things simple and avoid making extra changes, adding some risk for little benefit.

Adding indexedDb needs the following extra changes:

• open() the db earlier
• change the success/complete callbacks structure. Since those are sync, I couldn't DAO.store.set (async) in success, and make the complete callback to wait for the success callback to finish resolving its promises.
• if the database already exists, adding new keys to objectStores and then operating on them won't work until the db is recreated.

Looks like sessionStorage is not used as a mean for caching, but as a fallback in case the request fails. If the value is not there (first request), the program will fail anyway because it will be undefined.

With this patch, it would fail the same way, with the difference the "fallback" values will be lost on page reload, which looks like a rare scenario.
On the other hand, with indexedDb, the cached metadata will persist across reloads, tabs and future sessions - so the behavior will change as well in case the fallback is needed.

I went ahead and implemented an alternative fix leveraging indexedDb here: 2.40.4.1...EyeSeeTea:dhis2-core:fix/dataentry-sessionstorage-limit-idb

I created a separate dhis2.storage.Store instance to avoid the need for the user to remove the database first. This is to avoid the error IDBDatabase.transaction: 'metaData' is not a known object store name. Notice that DAO.store.set is async but also can throw sync Errors here.

Tested different scenarios and both approaches seem to work fine.

[tokland]

@MatiasArriola , Thanks for the thorough analysis! Indeed we have to be aware of the caching issues of introducing indexedDb.

[tokland]

This in-memory approach is fine for a simple approach if we don't want headaches with the cache.