Characters That Turn a Text Into Unicode

Updated

Short answer: One character outside the GSM-7 alphabet switches a whole text to UCS-2, the Unicode encoding, which cuts a single segment from 160 characters to 70. The characters to watch for are curly quotes, long dashes, the one-character ellipsis, non-standard spaces, emoji and some accented letters. A few symbols, such as { and ], stay in GSM-7 but count as two characters.

Why one character changes the whole message

Every phone and network has to support GSM-7. The 3GPP standard says "Implementation of the GSM 7 bit default alphabet is mandatory. Support of other character sets is optional." Anything GSM-7 can't encode moves the message to UCS-2. Twilio: "If you include any non GSM-7 characters in your message body, we will automatically fall back to UCS-2 encoding (which will limit message bodies to 70 characters each)."

The switch covers the whole message, so one character can change the price of a long text. Twilio's own example: "a body with 152 GSM-7-compatible characters and a single Unicode character will be split into 3 messages when encoded in UCS-2. This will incur charges for 3 outgoing messages against your account."

The same thing happens to everyday messages. This reminder is 142 characters:

Hi Maria, it’s Dan from Acme Roofing. Your free inspection is booked for Tue 3pm. Reply YES to confirm or call 555-0100. Reply STOP to opt out

With a straight apostrophe (') in "it's", it fits in 1 GSM-7 segment. With the curly apostrophe (’) shown above, it is UCS-2 and takes 3 segments. SMS Segments Explained covers the limits behind those numbers.

Characters that force UCS-2

CharacterWhat it isPlain GSM-7 version
“ and ”Curly double quotes"
‘ and ’Curly single quotes and apostrophes'
– and —En dash and em dash-
…Ellipsis typed as one character...
•Bullet-
Non-standard spacesCharacters that look like a space, or like nothingAn ordinary space, or nothing
EmojiPicturesNone
Chinese hanzi, Korean hangulCharacter-based scriptsNone
á, í, úAccented letters outside GSM-7The plain letter, if you accept the change

The plain versions in the first five rows are the ones Twilio's Smart Encoding feature uses. Twilio's list of characters it replaces has about 270 entries. Besides the ones above, it covers fullwidth forms, small capital letters and several Unicode spaces, such as U+00A0 and U+200B.

Accented letters are mixed. Twilio: "GSM 03.38 includes some accented characters such as ñ, à, and ö, but does not include others such as á, í, or ú." So "año" stays in GSM-7, but "está" switches the whole message to UCS-2.

Characters that count as two

GSM-7 has a second, smaller table, which the standard reserves "for symbols of international significance (e.g currency symbols)." Its characters stay in GSM-7 but cost two places each. Twilio: "Some characters, such as { and ], need an escape character. These escape characters count as two characters in a GSM-7-encoded message." The standard puts it this way: "the number of displayable characters will reduce by one for every instance where the GSM 7 bit default alphabet extension table is used."

The extension table in 3GPP TS 23.038 holds:

  • the caret ^, the tilde ~ and the backslash \
  • curly braces { } and square brackets [ ]
  • the vertical bar (pipe)
  • one currency symbol, which Twilio's segment calculator shows as the euro sign €
  • a page-break code

So a 159-character text with one [ in it counts as 160 and fits one segment. A 160-character text with one { counts as 161 and needs two.

There is one more catch. If a phone doesn't understand the escape code, the standard says it "shall display it as a space character."

Where these characters come from

  • Writing apps. Twilio warns that "Text editors designed for writing might automatically add angled smart quotes, non-standard spaces, or punctuation which looks similar to GSM-7 but is a different Unicode character."
  • AI-written copy. A 2026 Twilio post warns: "The dreaded em-dash (—) – a new favorite tool of AI-composed communications – can become a problem in your text if they aren't stripped clean."
  • Web pages. HighLevel describes "Hey there" copied from a web page that came to 4 segments, because it "contains many hidden (empty) characters".
  • Tools that hide them. Twilio said in 2017 that its console displays non-GSM spaces sent through the API "as regular U+0020 spaces", so you could not see them there.

How to find and fix them

  1. Paste the final text into a segment calculator that lists each character outside GSM-7.
  2. Swap curly quotes, long dashes, ellipses and bullets for the plain versions in the table above.
  3. Delete or retype spaces in any text you copied from a document or web page.
  4. Decide about characters that have no plain version, such as an emoji, an accented name or another script. If one stays, the message is UCS-2, so keep it within 70 characters or plan for 67 per part.
  5. Write templates where smart quotes are switched off, or check them again after every edit.

If you send through Twilio, Smart Encoding can make many of these swaps when the message is sent. Twilio Smart Encoding explains what it covers and where it stops.

Common mistakes

  • Assuming every accented letter is Unicode. ñ, à and ö are in GSM-7. á, í and ú are not.
  • Expecting Smart Encoding to fix a message with an emoji. Twilio says it "does not transliterate messages that contain emoji", so a curly quote in the same message stays too.
  • Judging by eye. Curly and straight apostrophes look alike, and odd spaces look like ordinary ones.
  • Forgetting the double-count symbols. Brackets, braces and the tilde take two places each, which matters in a text that sits close to 160.

Precheckly's free SMS segment calculator highlights each character outside GSM-7, explains what it is and replaces look-alike characters in one click, with undo. It runs entirely in your browser and sends nothing.

Sources

Not legal advice. Carrier rules and provider documentation change, so check the linked sources before you rely on them.