ECI收集的用户标准输出、错误日志是原生K8S落盘的日志,K8S会在每行日志前增加时间戳、来源等信息,这就破坏了用户原生的日志格式,比如用户的标准输出原生记录的是JSON格式,K8S添加前缀后JSON解析就会失败,如下:
2020-04-02T15:40:05.440500764+08:00 stdout F {"key1":"val1","key2":"val2"}
2020-04-02T15:40:07.442412564+08:00 stdout F {"key1":"val1","key2":"val2"}
2020-04-02T15:40:09.442774495+08:00 stdout F {"key1":"val1","key2":"val2"}
2020-04-02T15:40:11.443799303+08:00 stdout F {"key1":"val1","key2":"val2"}
2020-04-02T15:40:13.445099622+08:00 stdout F {"key1":"val1","key2":"val2"}
2020-04-02T15:40:15.445934358+08:00 stdout F {"key1":"val1","key2":"val2"}
2020-04-02T15:40:17.447064707+08:00 stdout F {"key1":"val1","key2":"val2"}
2020-04-02T15:40:19.448112987+08:00 stdout F {"key1":"val1","key2":"val2"}
2020-04-02T15:40:21.449393263+08:00 stdout F {"key1":"val1","key2":"val2"}
本文采用SLS Processor能力解决用户日志JSON解析的问题。
ECI的用户日志会收集到用账户下的日志仓库内,找到相应的logstore后修改配置,采用极简模式并启用插件处理能力,如下图:
插件配置内容如下: 参见sls-json-processor。
{
"processors": [
{
"type": "processor_anchor",
"detail": {
"SourceKey": "content",
"Anchors": [
{
"Start": "stdout F ",
"Stop": "",
"FieldName": "json_content",
"FieldType": "string",
"ExpondJson": false
}
]
}
},
{
"type": "processor_json",
"detail": {
"SourceKey": "json_content",
"KeepSource": false,
"ExpandConnector": ""
}
}
]
}
保存配置后隔几秒,就可以查看到正常解析的日志内容,如下所示:
可见,SLS已经正确解析了JSON日志。