<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[HandyNest]]></title><description><![CDATA[HandyNest]]></description><link>https://handynest.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Sat, 31 Oct 2026 17:48:12 GMT</lastBuildDate><atom:link href="https://handynest.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Your Word Counter Is Lying to You: How Unicode Breaks Naive Word Splitting]]></title><description><![CDATA[Open any "how to count words in JavaScript" tutorial and you will see the same one-liner: It works for "hello world". It falls apart the moment real text…]]></description><link>https://handynest.hashnode.dev/your-word-counter-is-lying-to-you-how-unicode-breaks-naive-word-splitting</link><guid isPermaLink="true">https://handynest.hashnode.dev/your-word-counter-is-lying-to-you-how-unicode-breaks-naive-word-splitting</guid><category><![CDATA[JavaScript]]></category><category><![CDATA[unicode]]></category><category><![CDATA[webdev]]></category><category><![CDATA[Tutorial]]></category><dc:creator><![CDATA[Muhammad Imran]]></dc:creator><pubDate>Sat, 24 Oct 2026 07:06:14 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6adc4a344f99182bb586b952/b4950ec0-ef07-4920-84cc-4ba8263f5385.webp" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Open any "how to count words in JavaScript" tutorial and you will see the same one-liner:</p>
<pre><code class="language-js">const count = text.trim().split(/\s+/).length;
</code></pre>
<p>It works for <code>"hello world"</code>. It falls apart the moment real text shows up. Emojis, Chinese characters, hyphenated compounds, contractions, and URLs each break a naive counter in their own quiet way, and the result is a number the user cannot trust. This post walks through the five ways word counting goes wrong and shows how to build a counter that gets it right.</p>
<h2>1. The whitespace trap: multiple spaces and newlines</h2>
<p>The basic version counts empty strings. <code>"hello  world".split(" ")</code> gives <code>["hello", "", "world"]</code>, length 3. Trimming and splitting on <code>/\s+/</code> fixes this specific case, so most tutorials stop here. But whitespace is the easy part.</p>
<h2>2. Hyphens and apostrophes: where one word ends</h2>
<p>Is "state-of-the-art" one word or four? Is "don't" one word or two? Every word processor has an opinion, and your users have one too. If you split on whitespace only, "state-of-the-art" counts as one. If you split on every non-letter, "don't" becomes two ("don" and "t"), which is clearly wrong.</p>
<p>A reasonable rule that matches what most editors do: treat apostrophes inside a word as part of the word, and treat hyphenated compounds as one word. A regex with Unicode property escapes handles this cleanly:</p>
<pre><code class="language-js">const wordPattern = /[\p{L}\p{N}]+(?:['\u2019-][\p{L}\p{N}]+)*/gu;
const words = text.match(wordPattern) ?? [];
</code></pre>
<p><code>\p{L}</code> matches any Unicode letter, not just a-z, so accented characters and non-Latin scripts are covered. The optional group keeps "don't", "l\u2019h\u00f4pital", and "state-of-the-art" as single tokens.</p>
<h2>3. Emojis and zero-width joiners</h2>
<p>Consider the family emoji: it looks like one character but it is actually seven code points joined by zero-width joiners (U+200D). A counter that splits on whitespace counts it as one word, which is fine. But a counter that tries to be clever and counts "characters" with <code>text.length</code> reports 11. Word counting and character counting are different jobs, and emoji expose every shortcut.</p>
<p>The rule: never count a zero-width joiner or a variation selector as a word boundary. If you use the regex above, emoji sequences are simply not letters, so they are skipped. That is the correct behavior for a word counter. If you also need an accurate character count, count grapheme clusters with <code>Intl.Segmenter</code>, not UTF-16 code units.</p>
<h2>4. CJK text has no spaces at all</h2>
<p>This is the big one. Chinese, Japanese, and Korean text does not use spaces between words. A whitespace-based counter reports an entire Chinese paragraph as one word. That is not a rounding error; it is completely wrong.</p>
<p>There is no perfect regex answer here, because word boundaries in CJK text require a dictionary or a segmentation model. But a widely used practical convention is to count each CJK character as one word:</p>
<pre><code class="language-js">const cjkPattern = /[\u4E00-\u9FFF\u3400-\u4DBF\u3040-\u30FF\uAC00-\uD7AF]/g;
const cjkCount = (text.match(cjkPattern) ?? []).length;
</code></pre>
<p>Combined with the letter-based matcher for spaced scripts, this gives a counter that behaves sensibly across scripts. It is an approximation, and you should say so in your UI, but it beats reporting "1 word" for a 500-character paragraph.</p>
<h2>5. URLs, emails, and numbers</h2>
<p>Should "<a href="https://example.com/some/path">https://example.com/some/path</a>" count as one word or five? Most counters say one, because splitting URLs on punctuation produces nonsense. The regex in section 2 already treats it as one token, since it only splits on characters outside the letter/number/apostrophe/hyphen set. Numbers like "3.14" or "1,000" stay intact too. Just make sure your character classes include <code>\p{N}</code> (any Unicode digit), or "2026" vanishes from the count.</p>
<h2>The modern shortcut: Intl.Segmenter</h2>
<p>If you target modern browsers, the platform now has a built-in answer:</p>
<pre><code class="language-js">const segmenter = new Intl.Segmenter("en", { granularity: "word" });
const words = [...segmenter.segment(text)].filter(s =&gt; s.isWordLike);
console.log(words.length);
</code></pre>
<p><code>Intl.Segmenter</code> handles locale-aware word boundaries, including scripts without spaces, far better than any hand-rolled regex. It is supported in all current browsers. The catch is older environments: if you need broad compatibility, keep the regex approach as a fallback and feature-detect the segmenter.</p>
<h2>Putting it together</h2>
<p>A robust client-side word counter is about 30 lines:</p>
<ol>
<li>Feature-detect <code>Intl.Segmenter</code> and use it when available.</li>
<li>Fall back to the Unicode-aware regex for letters, numbers, apostrophes, and hyphens.</li>
<li>Add the CJK character count for unspaced scripts.</li>
<li>Never use <code>String.length</code> for anything the user sees; count grapheme clusters if you show characters.</li>
</ol>
<p>If you would rather not build this yourself, there are free tools that already handle these edge cases. HandyNest has a free word counter at <a href="https://handynest.live/tools/word-counter.html">https://handynest.live/tools/word-counter.html</a> that runs entirely in the browser and deals with Unicode text, hyphenation, and CJK input the way this post describes.</p>
<p>Word counting looks like a solved problem until you test it with real text. Handle these five cases and your counter will agree with the number your users expect, which is the entire job.</p>
]]></content:encoded></item></channel></rss>