import re
from typing import Optional, Dict
# Regex: match one Nginx access-log line and extract ip, ts, method, path, status, rt_ms
NGINX_LOG_RE = re.compile(r"""
^
(?P<ip>(?:\d{1,3}\.){3}\d{1,3}) # IPv4 address (not range-validated)
\s+-\s+-\s+ # literal " - - " with spaces
\[
(?P<ts>\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2})
\]
\s+
(?P<method>[A-Z]+) # HTTP method in uppercase
\s+
(?P<path>\S+) # path without spaces
\s+
HTTP/\d\.\d # HTTP version like 1.1, 2.0
\s+
(?P<status>\d{3}) # 3-digit status code
\s+ # at least one space after status
(?:.*\s)? # optional middle fields (size, referrer, UA, etc.)
(?P<rt_ms>\d+)ms # integer response time followed by "ms"
$
""", re.VERBOSE)
def parse_nginx_line(line: str) -> Optional[Dict[str, str]]:
"""
Parse a single Nginx access-log line.
Returns a dict with keys: ip, ts, method, path, status, rt_ms, or None if no match.
"""
m = NGINX_LOG_RE.match(line)
if not m:
return None
return m.groupdict()
Detailed explanation
- ip: Captures an IPv4 dotted-quad. This pattern does not enforce 0–255 per octet for performance and simplicity.
- ts: Captures timestamp strictly as "YYYY-MM-DD HH:MM:SS" inside brackets.
- method: Captures one or more uppercase letters.
- path: Captures a non-space sequence (no spaces allowed).
- status: Captures a 3-digit HTTP status.
- rt_ms: Captures the trailing integer response time immediately followed by "ms".
- The regex is anchored with ^ and $ to ensure full-line matches and avoid partial matches.
- After status, the pattern tolerates arbitrary middle fields (bytes, referrer, user-agent, etc.) and extracts the final "ms" token at line end.
Usage examples and tests
def _show(line: str) -> None:
parsed = parse_nginx_line(line)
print(("OK", parsed) if parsed else ("NO MATCH", line))
# Positive examples (should match)
pos_lines = [
"203.0.113.12 - - [2024-08-12 14:23:11] GET /api/v1/items?id=42 HTTP/1.1 200 512 - Mozilla/5.0 18ms",
"198.51.100.99 - - [2025-01-01 00:00:00] POST /index.html HTTP/1.0 404 1024 \"-\" curl/8.4.0 123ms",
"192.0.2.5 - - [2024-12-31 23:59:59] HEAD /ping HTTP/1.1 200 18ms", # no extra middle fields
]
# Negative examples (should not match)
neg_lines = [
"203.0.113.12 - - [2024/08/12 14:23:11] GET /api/v1 HTTP/1.1 200 18ms", # wrong timestamp format
"203.0.113.12 - - [2024-08-12 14:23:11] get /api HTTP/1.1 200 18ms", # method not uppercase
"203.0.113.12 - - [2024-08-12 14:23:11] GET /api v1 HTTP/1.1 200 18ms", # space in path
"203.0.113.12 - - [2024-08-12 14:23:11] GET /api HTTP/1.1 2000 18ms", # 4-digit status
"203.0.113.12 - - [2024-08-12 14:23:11] GET /api HTTP/1.1 200 18 ms", # space between number and ms
"2001:db8::1 - - [2024-08-12 14:23:11] GET /api HTTP/1.1 200 18ms", # IPv6 not supported by this pattern
]
for line in pos_lines:
_show(line)
for line in neg_lines:
_show(line)
# Minimal assertion checks
assert parse_nginx_line(pos_lines[0]) == {
'ip': '203.0.113.12',
'ts': '2024-08-12 14:23:11',
'method': 'GET',
'path': '/api/v1/items?id=42',
'status': '200',
'rt_ms': '18',
}
assert parse_nginx_line(pos_lines[2]) == {
'ip': '192.0.2.5',
'ts': '2024-12-31 23:59:59',
'method': 'HEAD',
'path': '/ping',
'status': '200',
'rt_ms': '18',
}
assert parse_nginx_line(neg_lines[0]) is None
assert parse_nginx_line(neg_lines[1]) is None
assert parse_nginx_line(neg_lines[2]) is None
assert parse_nginx_line(neg_lines[3]) is None
assert parse_nginx_line(neg_lines[4]) is None
assert parse_nginx_line(neg_lines[5]) is None
Common matching scenarios
- ETL/streaming extraction: Use the compiled regex to parse each log line and emit a dict with the six fields for downstream processing.
- Batch parsing: Iterate over a file line by line, calling parse_nginx_line and filtering out None results.
- Validation: Quickly validate that an input string conforms to the expected log shape before ingestion.
Performance notes
- The pattern is anchored (^) and ($) to prevent partial matches and reduce backtracking.
- The only unbounded segment after the status uses a single greedy ".*" before the final whitespace and "rt_ms". This remains linear-time for typical single-line log entries.
- Octet range validation for IPv4 is intentionally omitted to avoid lengthy alternations. If strict 0–255 validation is required, consider validating the captured ip separately in Python for clarity and performance.
- Compile the regex once (as shown) and reuse it across lines to avoid recompilation overhead.
