Skip to content

fix(splunk-hec-logging): safely handle non-200 responses without text field - #14012

Merged
LiteSun merged 3 commits into
apache:masterfrom
bhuvan-somisetty:fix-splunk-hec-nil-concat
Nov 13, 2026
Merged

LiteSun merged 3 commits into
apache:masterfrom
bhuvan-somisetty:fix-splunk-hec-nil-concat

Conversation

@bhuvan-somisetty

@bhuvan-somisetty bhuvan-somisetty commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Description

In apisix/plugins/splunk-hec-logging.lua, send_to_splunk handles non-200 responses by decoding JSON and concatenating body.text. When Splunk HEC or an intermediate reverse proxy returns an error response where JSON decodes to a table without a string text field (such as {"message": "Unauthorized"}, {"text": null} where cjson.null is userdata, or non-table/nested values), concatenating body.text triggers a Lua runtime error (attempt to concatenate field 'text'), skipping batch processor retry and processed_entries accounting.

This PR updates send_to_splunk to check type(body) == "table" and type(body.text) == "string" before formatting "failed to send splunk, " .. body.text, and otherwise returns "failed to send splunk, http status code: " .. res.status .. ", body: " .. (res.body or ""), preserving both the HTTP status code and response body while keeping the existing Invalid authorization error format unchanged.

Which issue(s) this PR fixes:

Fixes #14011

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

Comment thread apisix/plugins/splunk-hec-logging.lua Outdated
if not body then
return false, "failed to send splunk, http status code: " .. res.status
local err_msg
if type(body) == "table" and (body.text or body.message) then

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI error: ERROR: apisix/plugins/splunk-hec-logging.lua: line 175: getting the Lua global "type". The local type should be replaced with a cached global, local type = type, so that lj-releng no longer reports this error.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated, thanks!

nic-6443
nic-6443 previously approved these changes Oct 26, 2026
… field

In `apisix/plugins/splunk-hec-logging.lua`, `send_to_splunk` expects error
responses to contain `body.text`. When Splunk HEC or an intermediate proxy
returns an error response with JSON lacking a `text` attribute, attempting
to concatenate `body.text` causes an unhandled runtime error:
`attempt to concatenate field 'text' (a nil value)`.

This commit falls back to `body.message`, `res.body`, or the HTTP status
code when `body.text` is not present, avoiding the crash.

Signed-off-by: bhuvan-somisetty <somisettybhuvan5@gmail.com>
Fixes lj-releng lint error by caching global `type`.

Signed-off-by: bhuvan-somisetty <somisettybhuvan5@gmail.com>
nic-6443
nic-6443 previously approved these changes Nov 11, 2026
@moonming

Copy link
Copy Markdown
Member

Thanks for the fix — the nil concat is real, and it's worse than a log line: since the batch processor doesn't pcall the callback, a throw skips both retry and the processed_entries accounting, so with the default max_pending_entries (8192, since 3.18) the worker eventually discards all splunk logs until reload, even after the endpoint recovers. I reproduced this on 3.19.0.

Two remaining cases still throw with this patch: {"text": null} (cjson.null is truthy userdata) and {"message": {...}} (table). Also, for non-JSON bodies (e.g. an HTML 502 from a proxy) the status code is now dropped. Suggest:

if type(body) == "table" and type(body.text) == "string" then
    return false, "failed to send splunk, " .. body.text
end
return false, "failed to send splunk, http status code: " .. res.status
              .. ", body: " .. (res.body or "")

This keeps the existing Invalid authorization assertion unchanged. A test case for {"text": null} would be good too. #14192 (draft) makes the batch processor pcall the callback, so this class of bug can't leak entries in any logger.

…/body fallback

Check `type(body.text) == "string"` so `{"text": null}` (cjson.null
userdata) does not throw on string concatenation, and include both HTTP
status code and raw response body in the fallback error message. Add
regression test coverage for `{"text": null}`.

Signed-off-by: bhuvan-somisetty <somisettybhuvan5@gmail.com>
@bhuvan-somisetty

Copy link
Copy Markdown
Contributor Author

Updated as suggested and added a test case for {"text": null}, thanks!

@membphis membphis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@LiteSun
LiteSun merged commit b941936 into apache:master Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: splunk-hec-logging crashes with nil concatenation when error response json lacks text field

7 participants