Expose a function to check whether a string is valid UTF-8 #20984

Activity

1. stof

   added

   Feature

   Status: Needs Triage

   on Jan 20, 2026
2. iluuu1994 commented on Jan 20, 2026

   

   iluuu1994

   on Jan 20, 2026

   Last edited by iluuu1994

   Member

   More actions

   Isn't that pretty much what mb_check_encoding($string, 'UTF-8') already does, apart from not being in core required? It checks the encoding and adds the IS_STR_VALID_UTF8 flag if appropriate, and avoids the check again in the future. Apart from some internal strings, this flag is only added by ext-mbstring when checking encodings on-demand. There's another feature request to add the flag for compile time strings (GH-10853), but that would still require a runtime encoding check for dynamic strings.

   Also related to https://wiki.php.net/rfc/unicode_text_processing.
3. stof commented on Jan 20, 2026

   

   stof

   on Jan 20, 2026

   Last edited by stof

   ContributorAuthor

   More actions

   mb_check_encoding($string, 'UTF-8') is indeed doing that, but this introduces a dependency on ext-mbstring which might not be OK for many packages (depending on what they do with the UTF-8 string, they might not need mbstring for that).

   In practice, many packages end up doing false !== preg_match('//u', $string) as a way to detect that, relying on the fact that PCRE fails for invalid UTF-8 when using the u modifier (and the fact that an empty regex should hopefully not fail for another weird reason, as it is a valid pattern and will not require backtracking that could reach the JIT limit).
4. stof commented on Jan 20, 2026

   

   stof

   on Jan 20, 2026

   ContributorAuthor

   More actions

   A quick github search for preg_match('//u' returns 7k results: https://github.com/search?q=%22preg_match%28%27%2F%2Fu%27%22&type=code

   And this includes only cases using a single-quoted string and the / delimiter for the regex.
5. mvorisek commented on Jan 20, 2026

   

   mvorisek

   on Jan 20, 2026

   Contributor

   More actions

   mb_check_encoding($string, 'UTF-8') is indeed doing that

   I can confirm, both are working great, the mb_check_encoding is very fast since PHP 8.2 - https://github.com/atk4/ui/blob/6.0.0/src/HtmlTemplate/Value.php#L28-L33
6. rr-it commented on Oct 18, 2026

   

   rr-it

   on Oct 18, 2026

   More actions

   Another option might be using iconv:

   @iconv('utf-8', 'utf-8//IGNORE', $str) === $str

   Quick search for iconv('UTF-8', 'UTF-8//IGNORE' on Github shows 25k results.

   ---

   AI says on this:

   $clean_string = iconv('UTF-8', 'UTF-8//IGNORE', $dirty_string);

   On many Linux servers, iconv() relies on the system's glibc library. When the source and target encodings are identical (UTF-8 to UTF-8), the library detects them as the same and skips the filtering process entirely. As a result, the invalid characters remain in your string.

   Workaround:
   To force iconv() to actually filter the text, you can convert the string to a different encoding (like UTF-16LE) and then immediately convert it back to UTF-8. This guarantees that the //IGNORE modifier takes effect.

   // Step 1: Convert to UTF-16 while ignoring invalid bytes
   $temp = iconv("UTF-8", "UTF-16LE//IGNORE", $dirty_string);
   
   // Step 2: Convert safely back to UTF-8
   $clean_string = iconv("UTF-16LE", "UTF-8", $temp);
7. staabm commented on Oct 19, 2026

   

   staabm

   on Oct 19, 2026

   Contributor

   More actions

   so would it make sense to short-circuit cases like preg_match('//u' to make use of IS_STR_VALID_UTF8 instead of working thru the regex engine?
8. mvorisek commented on Oct 19, 2026

   

   mvorisek

   on Oct 19, 2026

   Contributor

   More actions

   mb_* functions might be a good candicate to be frameless.