ruff UP012
labelled unknown
Analysis Nemotron 3 Super
Flags redundant `.encode("utf-8")` or `.encode(encoding="utf-8")` calls where UTF-8 is the default.
Fix: Replace `.encode("utf-8")` with `b"..."` for ASCII-only string literals, otherwise replace with `.encode()` (dropping the encoding argument).
1 hazards
- Rewriting `"\ud800".encode("utf-8")` to `"\ud800".encode()` changes behavior because surrogate code points raise UnicodeEncodeError without explicit error handling but are allowed when UTF-8 is implicit only if errors are not strict—wait, no: both use default errors='strict', so actually same? Let me correct: hazard is that removing explicit encoding may hide intent, but behavior identical if default is UTF-8 and errors not specified—but wait, the rule only fires when encoding is utf-8, so fix is safe regarding encoding. However: If the string literal contains non-ASCII and we replace with bytes literal, e.g., `"©".encode("utf-8")` → `b"©"` is invalid because bytes literal cannot contain © directly—it must be escaped or encoded. So hazard: rewriting to bytes literal for non-ASCII strings causes SyntaxError. Another: If the encode call is part of a larger expression where the bytes result is used in a context expecting a string, but no—both produce bytes. Wait, the fix for non-ASCII is to drop encoding arg, not to use bytes literal. The bytes literal substitution only applies when the string literal is ASCII-only. So hazard: incorrectly applying bytes literal substitution to a non-ASCII string literal (e.g., due to flawed ASCII detection) would produce invalid syntax. Another: If the string literal is ASCII but contains a backslash escape like `"\\x"`, the bytes literal `b"\\x"` is valid but represents different byte values? No, same. Hazard: comments attached to the encode call may be lost during rewrite if the tool does not preserve trivia. Another: If the encode call is on a variable that shadows a built-in or has a property named encode, but the rewrite assumes it's str.encode—if the variable is not a string, `.encode()` may fail or call a different method, same as before. Actually, same behavior. Hazard: In Python 2, UTF-8 is not default, but rule likely only active in Py3+ contexts—still, if run in Py2, removing encoding arg changes behavior. But assuming target is Py3. Another: If the call uses keyword argument like `.encode(encoding="utf-8")`, rewriting to `.encode()` may break if there are subsequent positional arguments relying on keyword order—but encode only has two params: encoding and errors. If errors is passed positionally after, e.g., `.encode("utf-8", "replace")`, fixing to `.encode("replace")` would be wrong—but rule only fires when encoding is utf-8 AND no errors arg? Wait, documentation says flags unnecessary explicit UTF-8 encoding arguments. Examples show `.encode("utf-8")` and `.encode(encoding="utf-8")`. It does not mention cases with errors arg. So likely rule only fires when encoding is utf-8 and errors is not specified (or is default?). Actually, pyupgrade's UP012: it removes encoding=utf-8 even if errors is present? Let me check: pyupgrade docs say it removes encoding=utf-8 and optionally errors if default. But rule description here says: 'unnecessary calls to encode as UTF-8 and unnecessary explicit UTF-8 encoding arguments.' Examples don't show errors. To be safe, assume rule only fires when encoding is utf-8 and no errors argument is provided. Therefore, fix is to remove encoding arg entirely. Hazard: if errors argument is present but rule incorrectly fires (shouldn't), then removing encoding but leaving errors would be `.encode(errors="strict")` which is invalid because encoding is required unless it's first param? Actually, str.encode(encoding='utf-8', errors='strict') -> if we remove encoding, we get .encode(errors='strict') which passes errors as positional first arg, i.e., encoding becomes the errors value—wrong. So hazard: if the rule mistakenly fires on calls that also have an errors argument (which it shouldn't per spec, but if implementation flaw), the fix breaks by misordering arguments. Another: If the string literal is ASCII but the bytes literal is used in a context that decodes it with a different encoding, but no—bytes are bytes. Another: If the encode call is chained, e.g., `"text".encode("utf-8").decode()`, fix to `b"text".decode()` is same. Another: If the literal contains only ASCII but uses escapes that represent non-ASCII in bytes? No, ASCII literals by definition are 0-127. Hazard: Source file encoding declaration matters—if file is not UTF-8, but Python 3 defaults source to UTF-8 anyway. Another: If the encode call is inside a lambda or comprehension and the fix alters indentation or line breaking, causing syntax error—unlikely but possible if tool mishandles multiline. Another: If the call is on a string literal that is empty, `"".encode("utf-8")` → `b""` is fine. Another: If the string literal is ASCII but contains a null byte via escape, `"\x00".encode("utf-8")` → `b"\x00"` same. I'll curate 8-12 specific hazards.
Programs
| id | result | title | hazard |
|---|---|---|---|
| r1-p001 | silent prediction off | Shadowed built-in str | If a local variable named 'str' shadows the built-in and has an encode method with different behavior, the rule's fix assumes str.encode but the attribute access resolves to the local variable. |
| r1-p002 | fix ok | Dict ordering and duplicate keys | The rule fires inside a dict literal; if the fix alters spacing or line breaks, it could change how duplicate keys are resolved in Python 3.6+ where dict preserves insertion order. |
| r1-p003 | fix ok | Pathlib vs os.path on relative paths | The encoded bytes result is used to create a file via os.fsdecode; if the fix changes the byte sequence due to incorrect ASCII assumption, decoding fails or yields wrong path. |
| r1-p004 | silent | Bytes vs str with newlines | The rule replaces .encode('utf-8') with a bytes literal; if the literal contains \n and is later decoded with universal newlines enabled, behavior differs from explicit encoding. |
| r1-p005 | fix ok | Comments attached to encode call | If the tool's fix does not preserve trailing comments, the comment is lost, which may affect tools that parse comments for metadata. |
| r1-p006 | silent prediction off | Generator expression consuming encode result | The encode call appears in a generator expression; if the fix alters evaluation timing or raises an exception due to faulty bytes literal substitution, lazy evaluation breaks. |
| r1-p007 | fix ok | Non-ASCII string literal - surrogate hazard corrected | If the rule incorrectly applies bytes-literal substitution to a non-ASCII literal due to flawed ASCII detection (e.g., counting code units instead of characters), it produces a SyntaxError. |
| r1-p008 | fix ok | String formatting with % operator | The encode result is used in % formatting; if the fix changes the byte content (e.g., due to incorrect literal substitution), the formatted output changes. |
| r2-p001 | silent prediction off | Truthiness hazard | truthiness of custom __bool__/__len__, None vs falsy, 'is' vs '==' |
| r2-p002 | fix ok | Generator consumption hazard | generators vs lists, iterators consumed once, lazy vs eager evaluation |
| r2-p003 | silent prediction off | Keyword-only forwarding hazard | keyword-only, positional-only, *args/**kwargs forwarding, defaults evaluated once |
| r2-p004 | silent prediction off | Bytes vs str mode hazard | bytes vs str, encoding and newline handling, text vs binary modes |
| r2-p005 | silent prediction off | Descriptor side-effect hazard | class attributes vs instance attributes, descriptors, properties with side effects |
| r2-p006 | silent prediction off | Dict ordering/hash hazard | dict ordering, duplicate keys, __hash__/__eq__ overrides, sets |
| r2-p007 | fix ok | Async generator hazard | async def/await ordering, async generators |
| r2-p008 | silent prediction off | Context manager write-only hazard | context managers that are not files, objects with write() but no close() |
Sources
r1-p001 · Shadowed built-in str
1def f():
2 str = type('S', (), {'encode': lambda self, *a: b'local'})()
3 x = str.encode('utf-8')
4 print(x)
5f()
b'local'
r1-p002 · Dict ordering and duplicate keys
1d = {
2 'a': 'hello'.encode('utf-8'),
3 'b': 'world'.encode('utf-8'),
4 'a': 'duplicate'.encode('utf-8'),
5}
6print(list(d.values()))
--- orig.py 2026-10-17 17:16:34.130126296 -0400 +++ fixed.py 2026-10-17 17:16:34.304114768 -0400 @@ -1,6 +1,6 @@ d = { - 'a': 'hello'.encode('utf-8'), - 'b': 'world'.encode('utf-8'), - 'a': 'duplicate'.encode('utf-8'), + 'a': b'hello', + 'b': b'world', + 'a': b'duplicate', } print(list(d.values()))
[b'duplicate', b'world']
r1-p003 · Pathlib vs os.path on relative paths
1import os
2path_bytes = 'foo/bar'.encode('utf-8')
3try:
4 os.makedirs(path_bytes, exist_ok=True)
5except TypeError:
6 pass # bytes not accepted directly in some versions
7finally:
8 decoded = os.fsdecode(path_bytes)
9 print('decoded:', decoded)
--- orig.py 2026-10-17 17:16:34.399108473 -0400 +++ fixed.py 2026-10-17 17:16:34.554098203 -0400 @@ -1,5 +1,5 @@ import os -path_bytes = 'foo/bar'.encode('utf-8') +path_bytes = b'foo/bar' try: os.makedirs(path_bytes, exist_ok=True) except TypeError:
decoded: foo/bar
r1-p004 · Bytes vs str with newlines
1s = 'a\nb'
2encoded = s.encode('utf-8') # rule does not fire: no explicit utf-8
3print('encoded:', encoded)
encoded: b'a\nb'
r1-p005 · Comments attached to encode call
1value = 'text'.encode('utf-8') # encoding marker
2print(value)
--- orig.py 2026-10-17 17:16:34.788082699 -0400 +++ fixed.py 2026-10-17 17:16:34.960071303 -0400 @@ -1,2 +1,2 @@ -value = 'text'.encode('utf-8') # encoding marker +value = b'text' # encoding marker print(value)
b'text'
r1-p006 · Generator expression consuming encode result
1gen = (c.encode('utf-8') for c in 'ab')
2print(list(gen))
[b'a', b'b']
r1-p007 · Non-ASCII string literal - surrogate hazard corrected
1try:
2 # This should trigger the rule but NOT be replaced with bytes literal
3 result = '♥'.encode('utf-8')
4 print(result)
5except SyntaxError as e:
6 print('SyntaxError:', e)
--- orig.py 2026-10-17 17:16:35.193055889 -0400 +++ fixed.py 2026-10-17 17:16:35.363044646 -0400 @@ -1,6 +1,6 @@ try: # This should trigger the rule but NOT be replaced with bytes literal - result = '♥'.encode('utf-8') + result = '♥'.encode() print(result) except SyntaxError as e: print('SyntaxError:', e)
b'\xe2\x99\xa5'
r1-p008 · String formatting with % operator
1template = b'Result: %s'
2encoded = 'ok'.encode('utf-8')
3output = template % encoded
4print(output)
--- orig.py 2026-10-17 17:16:35.461038164 -0400 +++ fixed.py 2026-10-17 17:16:35.627027186 -0400 @@ -1,4 +1,4 @@ template = b'Result: %s' -encoded = 'ok'.encode('utf-8') +encoded = b'ok' output = template % encoded print(output)
b'Result: ok'
r2-p001 · Truthiness hazard
1
2def check(x):
3 return x == b''
4
5class Falsy:
6 def __bool__(self): return False
7 def __len__(self): return 0
8
9obj = Falsy()
10s = '\\xe9'
11encoded = s.encode('utf-8') # ruled by UP012
12print(f'{check(obj) = } {check(None) = } {check(encoded) = }')
13if encoded == b'':
14 print('empty')
15else:
16 print(f'non-empty: {encoded}')
check(obj) = False check(None) = False check(encoded) = False non-empty: b'\\xe9'
r2-p002 · Generator consumption hazard
1
2gen = (c for c in 'café'.encode('utf-8')) # UP012 triggers here
3first = list(gen)
4second = list(gen) # exhausted!
5print(f'{len(first)} {len(second)}')
6if len(second) == 0:
7 print('generator exhausted after first use')
8else:
9 print('unexpected: generator reusable')
--- orig.py 2026-10-17 17:16:53.140887379 -0400 +++ fixed.py 2026-10-17 17:16:53.307876691 -0400 @@ -1,5 +1,5 @@ -gen = (c for c in 'café'.encode('utf-8')) # UP012 triggers here +gen = (c for c in 'café'.encode()) # UP012 triggers here first = list(gen) second = list(gen) # exhausted! print(f'{len(first)} {len(second)}')
5 0 generator exhausted after first use
r2-p003 · Keyword-only forwarding hazard
1
2def process(data, *, encoding='utf-8', errors='strict'):
3 return data.encode(encoding, errors)
4
5wrapper = lambda s: process(s, encoding='utf-8') # UP012 triggers on inner call
6result = wrapper('naïve')
7print(f'{result}')
8try:
9 wrapper('naïve').decode()
10except Exception as e:
11 print(f'{type(e).__name__}: {e}')
b'na\xc3\xafve'
r2-p004 · Bytes vs str mode hazard
1
2text = 'hello\nworld'
3encoded = text.encode('utf-8') # UP012 triggers
4with open('test.bin', 'wb') as f:
5 f.write(encoded)
6with open('test.bin', 'rb') as f:
7 raw = f.read()
8print(f'{raw == encoded}')
9with open('test.bin', 'r') as f: # text mode
10 decoded = f.read()
11print(f"{decoded == 'hello\\nworld'} # text-mode decoding may differ"
12)
True False # text-mode decoding may differ
r2-p005 · Descriptor side-effect hazard
1
2class EncodingTracker:
3 def __get__(self, obj, owner):
4 if obj is None:
5 return self
6 obj._last_encoded = 'utf-8'
7 return b'test'
8
9class Holder:
10 encoded = EncodingTracker()
11
12h = Holder()
13print(f'before: {getattr(h, "_last_encoded", None)}')
14res = h.encoded # triggers descriptor, not UP012
15s = 'café'
16encoded_s = s.encode('utf-8') # UP012 triggers here
17print(f'after: {h._last_encoded} {encoded_s}')
before: None after: utf-8 b'caf\xc3\xa9'
r2-p006 · Dict ordering/hash hazard
1
2class BadKey:
3 def __init__(self, val): self.val = val
4 def __hash__(self): return hash(self.val)
5 def __eq__(self, other): return self.val == other.val
6
7k1 = BadKey('a')
8k2 = BadKey('a') # same hash, equal
9d = {}
10d[k1.encode('utf-8')] = 'first' # UP012 triggers on k1.encode()
11d[k2.encode('utf-8')] = 'second' # second triggers too
12print(f'{len(d)} {list(d.values())}')
13if len(d) == 1:
14 print('keys collided')
15else:
16 print('keys separated')
r2-p007 · Async generator hazard
1
2async def gen():
3 yield 'hi'.encode('utf-8') # UP012 triggers
4
5async def main():
6 g = gen()
7 first = [x async for x in g]
8 try:
9 second = [x async for x in g] # reused generator
10 except StopAsyncIteration:
11 second = []
12 print(f'{len(first)} {len(second)}')
13
14import asyncio
15asyncio.run(main())
--- orig.py 2026-10-17 17:16:54.141823331 -0400 +++ fixed.py 2026-10-17 17:16:54.431804804 -0400 @@ -1,6 +1,6 @@ async def gen(): - yield 'hi'.encode('utf-8') # UP012 triggers + yield b'hi' # UP012 triggers async def main(): g = gen()
1 0
r2-p008 · Context manager write-only hazard
1
2class LogSink:
3 def __init__(self):
4 self.buffer = b''
5 def write(self, data):
6 self.buffer += data
7 def flush(self):
8 pass
9
10sink = LogSink()
11msg = 'alert: ∆'
12encoded = msg.encode('utf-8') # UP012 triggers
13sink.write(encoded)
14sink.write(b'\n')
15print(f'{sink.buffer}')
16if b'alert:' in sink.buffer:
17 print('log captured')
18else:
19 print('log missing')
b'alert: \xe2\x88\x86\n' log captured