Sorting ls output respecting locale but not ignoring nonalphanumeric characters

On a GNU system, appending NULs to non-alphas may help:

$ ls | sed 's/[^[:alpha:]]/&\x0/g' | sort | tr -d '\0'
_b
a
c
č
d

That assumes file names don't contain newline characters. Generally, you can't sort lists of file names with sort as file names can very well be made of several lines themselves.

You could replace the newlines within filenames with / here before sorting. With zsh:

print -rNC1 -- *(N) | # print NUL-delimited
  tr '\n\0' '/\n' |
  sed 's/[^[:alpha:]]/&\x0/g' |
  sort |
  tr -d '\0' |
  tr '/' '\n'

Or to keep the list NUL-delimited so it can be post-processed:

print -rNC1 -- *(N) | # print NUL-delimited
  tr '\n\0' '/\n' |
  sed 's/[^[:alpha:]]/&\x0/g' |
  sort |
  tr -d '\0' |
  tr '/\n' '\n\0'

The strcoll() API that is used for sorting takes two NUL-terminated strings. Traditional sort implementations only support text input and text input precludes NULs so they're fine. GNU sort however, like GNU implementations of most standard text utilities do support NULs with their input.

I don't know how exactly GNU sort handles lines with NULs, but my guess would be that it breaks the lines on NULs and compares segments one to one. So, when comparing foo_\0car with foobar_\0more for instance, foo_ is compared with foobar_ and comes first.

You can also define the order of zsh globs using arbitrary transformations with the oe (order based on the evaluation of some code) or o+function glob qualifiers. But before calling strcoll(), zsh removes the NULs, so you can't use the same transformation as used for GNU sort above.

Instead, there, you could add 0s before sequences of non-alphas and 1s before sequences of alphas:

In ~/.zshrc

set -o extendedglob
mysort() {
  REPLY=${REPLY//(#m)[^[:alpha:]]##/0$MATCH}
  REPLY=${REPLY//(#m)[[:alpha:]]##/1$MATCH}
}

Then:

$ print -rC1 -- *(No+mysort)
_b
a
c
č
d