🔤 Section 12 · Question #18

How to Properly Convert String to byte[] and Back

Converting a string into a byte array (String $\to$ byte[]) and reconstructing a string from bytes (byte[] $\to$ String) are fundamental operations across networking, file I/O,...


🟢 Junior Level

Converting a string into a byte array (String $\to$ byte[]) and reconstructing a string from bytes (byte[] $\to$ String) are fundamental operations across networking, file I/O, database persistence, and cryptography.

The Golden Rule

ALWAYS explicitly specify the encoding using constants from java.nio.charset.StandardCharsets! Never call conversion methods without an explicit charset argument.

import java.nio.charset.StandardCharsets;

String original = "Hello, World! 🚀";

// 1. Correct encoding: String -> byte[]
byte[] bytes = original.getBytes(StandardCharsets.UTF_8);

// 2. Correct decoding: byte[] -> String
String restored = new String(bytes, StandardCharsets.UTF_8);

System.out.println(restored); // "Hello, World! 🚀"

Why Parameterless str.getBytes() is Banned in Production

Methods without an explicit Charset parameter rely on the platform’s default encoding (Charset.defaultCharset()). Prior to Java 18, Windows defaulted to Windows-1252 (or Cp1251), while Linux defaulted to UTF-8. Consequently, a file or network payload generated on Windows produced unreadable garbled text (Mojibake) when decoded on a Linux server.


🟡 Middle Level

Encoding Specification Styles Compared

Java supports three ways to specify a character set, but only one is considered clean enterprise practice:

// ❌ 1. Obsolete and Error-Prone (String Literal):
// Requires catching checked UnsupportedEncodingException!
// Incurs dynamic HashMap lookup on every call.
byte[] b1 = str.getBytes("UTF-8"); 

// 🟡 2. Dynamic Lookup:
// Avoids checked exceptions, but still burns CPU cycles on cache lookup.
byte[] b2 = str.getBytes(Charset.forName("UTF-8"));

// ✅ 3. Recommended Best Practice (StandardCharsets):
// Direct reference to pre-allocated singleton constant. Zero lookups, zero allocations, compile-time safe.
byte[] b3 = str.getBytes(StandardCharsets.UTF_8);
String s3 = new String(b3, StandardCharsets.UTF_8);

String Length vs. Byte Array Length

A frequent pitfall is assuming that string character length equals byte array length:

String text = "Привет";

System.out.println(text.length());                                // 6 (characters/code units)
System.out.println(text.getBytes(StandardCharsets.UTF_8).length);  // 12 (bytes!)

In UTF-8, ASCII characters take 1 byte, Cyrillic characters take 2 bytes each, and emojis require 4 bytes each. Network buffers and database column sizes must always be calculated against byte[].length, never str.length().

Why Raw Binary Data Must Never Be Converted to String

If you are handling raw binary data (e.g., a 32-byte SHA-256 hash, an AES key, image bytes, or compressed zlib streams), never convert them to String using new String(bytes, StandardCharsets.UTF_8):

  • UTF-8 contains invalid byte sequences (such as standalone bytes 0xFF, 0xFE, or orphaned continuation bytes). The decoder silently replaces them with \uFFFD.
  • A subsequent getBytes() call will emit the replacement bytes, permanently corrupting the cryptographic hash or binary structure.

Correct Text Representations for Binary Data:

  1. Base64 (Standard for JSON, HTTP headers, and tokens):
    String base64 = Base64.getEncoder().encodeToString(binaryBytes);
    byte[] restored = Base64.getDecoder().decode(base64);
    
  2. Hexadecimal String (Hex Format, Java 17+):
    String hex = HexFormat.of().formatHex(binaryBytes);
    byte[] restored = HexFormat.of().parseHex(hex);
    

🔴 Senior Level

GC Allocation Pressure in High-Throughput Pipelines

Every invocation of str.getBytes() or new String(bytes) incurs substantial heap overhead:

  1. Validates the charset and calculates target buffer length.
  2. Allocates fresh heap arrays: getBytes() allocates a new byte[] array, while new String() allocates the String object header plus the internal backing array.
  3. Transcodes and copies bytes via System.arraycopy or SIMD intrinsics.

In high-throughput services (100,000+ RPS), continuous conversions create heavy Young Generation GC pressure, causing frequent minor GC pauses.

Highload & Zero-Copy Patterns:

  • Netty ByteBuf / Java NIO ByteBuffer: Parse protocol frames and headers directly within byte buffers without materializing intermediate String objects.
  • Streaming Readers/Writers: Stream files without loading entire byte arrays into heap memory:
    try (BufferedReader reader = Files.newBufferedReader(path, StandardCharsets.UTF_8)) {
        String line;
        while ((line = reader.readLine()) != null) { /* process line */ }
    }
    

Handling Multi-Byte Boundary Splits in Socket Streams

In network programming (TCP/Socket streams), packets arrive in chunks. If a multi-byte UTF-8 character (e.g., Cyrillic 'Д' = 0xD0, 0x94) is split across two TCP packets (Packet 1 ends with 0xD0, Packet 2 begins with 0x94):

  • Calling new String(packet1, UTF_8) replaces 0xD0 with \uFFFD.
  • Calling new String(packet2, UTF_8) similarly corrupts 0x94.

Solution: Stateful CharsetDecoder:

CharsetDecoder decoder = StandardCharsets.UTF_8.newDecoder()
    .onMalformedInput(CodingErrorAction.REPORT)
    .onUnmappableCharacter(CodingErrorAction.REPORT);

ByteBuffer in = ByteBuffer.allocate(1024);
CharBuffer out = CharBuffer.allocate(1024);

// The decoder buffers incomplete byte sequences across chunks until next read
decoder.decode(in, out, false); // endOfInput = false indicates more data coming

Security Considerations: Passwords and Heap Retention

Converting passwords or secrets into String creates a security exposure:

  • Strings are immutable and cannot be overwritten. They remain in the Java heap until collected and swept by GC, leaving them vulnerable to extraction via heap dumps or /proc/$PID/mem memory scrapers.
  • Best Practice: Keep secrets in primitive char[] or byte[] arrays, and wipe them with zeroes immediately after use:
    byte[] secretBytes = readSecretFromNetwork();
    try {
        authenticate(secretBytes);
    } finally {
        Arrays.fill(secretBytes, (byte) 0); // Deterministic zeroing
    }
    

4 Tricky Questions

1. Why is StandardCharsets.UTF_8 preferred over str.getBytes("UTF-8") and Charset.forName("UTF-8")?

Answer:

  1. Elimination of Checked Exceptions: str.getBytes("UTF-8") forces developers to catch UnsupportedEncodingException, adding boilerplate for a charset that is guaranteed to be available in every conforming JVM.
  2. Lookup Performance: String literal lookups require the JVM to perform a synchronized hash map query inside Charset.availableCharsets() on every call.
  3. Type Safety & Zero Allocations: StandardCharsets.UTF_8 is a statically initialized final singleton. Passing it requires zero map lookups, zero heap allocations, and guarantees compile-time correctness.

2. What does new String(new byte[] {(byte) 0xD0}, StandardCharsets.UTF_8) return, and how do you enforce strict validation?

Answer: The standard new String(bytes, charset) constructor uses the CodingErrorAction.REPLACE policy by default. Because 0xD0 is the leading byte of a 2-byte UTF-8 sequence and the second byte is missing, the constructor silently replaces the incomplete byte with the Unicode replacement character \uFFFD () without throwing any exception. To enforce strict validation that throws an exception upon encountering corrupted bytes, use an explicit CharsetDecoder:

CharsetDecoder decoder = StandardCharsets.UTF_8.newDecoder()
    .onMalformedInput(CodingErrorAction.REPORT);
CharBuffer result = decoder.decode(ByteBuffer.wrap(bytes)); // Throws MalformedInputException

3. Why must you never convert a SHA-256 digest byte array to String via new String(hashBytes, StandardCharsets.UTF_8)?

Answer: A SHA-256 digest produces 32 pseudo-random bytes spanning arbitrary values from 0x00 to 0xFF. In UTF-8, certain byte values (such as 0xF5 through 0xFF or orphaned continuation bytes 0x80 to 0xBF) are syntactically invalid. The UTF-8 decoder replaces all invalid sequences with \uFFFD. Attempting to convert the resulting string back via str.getBytes() produces replacement bytes rather than the original digest, corrupting the hash. Binary digests must be encoded using Base64 or Hexadecimal (HexFormat).

4. What performance advantages does java.util.HexFormat (introduced in Java 17) offer over String.format("%02x")?

Answer:

  • String.format("%02x", b) is notoriously slow: it parses the format string at runtime, uses reflection, instantiates a java.util.Formatter object, and autoboxes primitive byte values into Byte heap instances.
  • HexFormat (Java 17+) is designed for high-performance vectorized formatting. It is immutable, thread-safe, uses direct bitwise shifts (>> 4) and pre-calculated lookup tables, executes orders of magnitude faster, and generates minimal heap allocations.

🎯 Interview Cheat Sheet

30-Second Summary

“To convert a String to byte[], use str.getBytes(StandardCharsets.UTF_8); to convert back, use new String(bytes, StandardCharsets.UTF_8). Calling conversion methods without specifying a charset relies on the OS default and leads to cross-platform data corruption. Never convert raw binary data (hashes, encryption keys) directly to String—use Base64 or HexFormat (Java 17+). For security, never store sensitive passwords in String because they cannot be cleared from memory; keep them in byte[] or char[] and zero them with Arrays.fill().”

Key Takeaways for Senior Interviews

  • StandardCharsets: Always use StandardCharsets.UTF_8 to eliminate checked exceptions and runtime charset lookups.
  • Length Disparity: Character length (str.length()) is not byte count (Cyrillic = 2 bytes/char, emojis = 4 bytes/char in UTF-8).
  • Binary Data: Binary payloads must be formatted via Base64 or HexFormat, never via character encodings.
  • Socket Chunking: Multi-byte characters split across network packets must be decoded using stateful CharsetDecoder with endOfInput = false.
  • Password Security: Sensitive credentials should remain in primitive arrays (byte[] / char[]) and immediately wiped using Arrays.fill(bytes, (byte) 0).

Red Flags to Avoid

  • ❌ Calling str.getBytes() without a Charset argument.
  • ❌ Using string literals like "UTF-8" and catching UnsupportedEncodingException.
  • ❌ Converting raw encrypted bytes or file binaries into String using charset decoders.
  • ❌ Assuming 1 character always translates to 1 byte in UTF-8.
  • ❌ Storing unhashed user passwords in immutable String instances.