Skip to content

Retire the python2 workarounds for inefficient listdir, scandir and walk#424

Merged
jonasbardino merged 3 commits intonextfrom
retire/python2-scandir-walk-and-listdir-workarounds
Jan 28, 2026
Merged

Retire the python2 workarounds for inefficient listdir, scandir and walk#424
jonasbardino merged 3 commits intonextfrom
retire/python2-scandir-walk-and-listdir-workarounds

Conversation

@jonasbardino
Copy link
Contributor

@jonasbardino jonasbardino commented Jan 27, 2026

Retire the python2 workarounds for inefficient listdir, scandir and walk now available by default in the os module.

Quite a few other modules still import those functions from fileio, where we used to automatically expose the most efficient versions on all python versions. So we leave them exposed in fileio, just without any such workarounds now that the native os version has the optimizations.

@jonasbardino jonasbardino self-assigned this Jan 27, 2026
@jonasbardino jonasbardino added the enhancement New feature or request label Jan 27, 2026
@jonasbardino jonasbardino changed the title Retire the python2 workarounds for inefficient walk and listdir Retire the python2 workarounds for inefficient listdir, scandir and walk Jan 27, 2026
@jonasbardino
Copy link
Contributor Author

jonasbardino commented Jan 27, 2026

The lint errors in sanity checks here initially were old and covered by #338 / #426, so they have now been eliminated with the merge of #426 .

@jonasbardino jonasbardino force-pushed the retire/python2-scandir-walk-and-listdir-workarounds branch from 74bdf51 to 087fa46 Compare January 27, 2026 18:11
…s` module.

There is quite a bit of related calling code to update acordingly to completely
eliminate the construct.
…lpers now

that we just use efficient os.X versions everywhere.
@jonasbardino jonasbardino force-pushed the retire/python2-scandir-walk-and-listdir-workarounds branch from 087fa46 to 18659ff Compare January 28, 2026 12:14
@jonasbardino jonasbardino marked this pull request as ready for review January 28, 2026 12:20
@jonasbardino
Copy link
Contributor Author

I know this one overlaps with #425 so please review with that in mind and/or get back to me to sync @albu-diku .
My gut-feeling is that we probably want to keep os.{listdir,scandir,walk} interfaced through fileio even though we no longer workaround the actual backend implementation for efficiency. To keep any future I/O backend replacement (e.g. block storage, transparent replication, etc.) open, I mean.

@jonasbardino jonasbardino requested a review from a team January 28, 2026 12:26
Copy link
Contributor

@Martin-Rehr Martin-Rehr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved

@jonasbardino jonasbardino merged commit c71b209 into next Jan 28, 2026
14 checks passed
@jonasbardino jonasbardino deleted the retire/python2-scandir-walk-and-listdir-workarounds branch January 28, 2026 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants