如何设置CentOS Dolphin系统监控报警，轻松实现实时掌握系统健康状况？

一、 ：运维人的“隐形痛点”，你中招了吗？

凌晨三点被

主要痛点直击：

• “盲区多”只监控OS不监控DB内核指标，等于裸奔。
• “响应慢”依赖人工巡检或简单阈值报警。误报漏报频发，真故障却推送不到人。
• “链路断”程序指标、数据库指标、业务指标割裂。排查耗时长，MTTR居高不下。说起来，

二、 基石篇：CentOS 操作程序层“体检”与即时感知

2.1 命令行急救包：临时排查与日常巡检必备

>//>//>-d>-d>-d>-d>-d>-d>-d>-d>-d>-d>//>////////> bash <

维度	主要命令/工具	适用场景 & 关键指标解读
	/	CPU核数持续高位 = 排队严重。
		0="内存交换；us+sy" cpu核数="CPU瓶颈；si/so" ≠=""> 80% = CPU饱和。 >
	/	0 需警惕 Swappiness 配置。
	td>	分析进程实际物理内存，排查 Dolphin 内存泄漏或 Off-heap 溢出。td>
	td>	历史内存趋势回溯。td>
td>/>/>/>/>/>//>/>/>/>/ 饱和;await>> svctm -> 排队严重;aqu-sz 队列长度，关注 WAL 日志盘 和 数据盘。/>/>

2.2 本地即时告警：别让故障“过夜”

echo "触发阈值" | mail -s "Alert" logger -t custommonitor "Disk Full" systemctl status dolphindb --no-pager journalctl-u dolphindbsince"-5 min" * * * * */root/scripts/checkdisk.shcheckmem.shcheckload.sh checkdisk.sh df-hP|awk'$5+{print$6$5}' memfree=$}') )&&echo"Mem low" load=$ cpucore=$ )&&echo"Load high" chmod+x*.sh crontab-e wallmail/logger/systemdcron sysstatenmonNetdataPromeusNodeExporterGrafanaZabbixNagiosPromeusAlertmanagerEmailWeChatDingTalkWebhookNodeExporterPromeusGrafanaAlertmanagerPromQLPromQLNodeExporterPromQLNodeExporterNodeExporterNodeExporterNodeExporterNodeExporterNodeExporterNodeExporterNodeExporterNodeExporterNodeExporterNodeExporterNodeExporterscrapeconfigsjobnamestaticconfigstargetsalertingalertmanagersstaticconfigstargetsalertgroupsnamrulesalertforexprforlabelsannotationssummarydescriptiongeneratorURLrunbookurlreceiverreceiversnameemailconfigstofromsmarthostauthusernameauthpasswordauthidentityrequiretlsreceiversnamewechatconfigscorpidapisecretagentidtopartywebhookconfigsurlsendresolvedglobalresolveroutetimerepeatintervalroutegroupbyreceivergroupwaitgroupintervalrepeatintervalroutesmatchreceivercontinuematchreceivercontinueinhibitrulestargetmatchsourcematchtargetmatchreequaloptsourcematchreequaloptmatchersmatchersmatchersmatchersmatchersmatchersmatchersmatchersmatchersmatchersmatchersmatchersmatchersmatchersmatchersmatchersalertmanageryamldolphindbdolphinschedulerdolphindbdolphindbdolphindbdolphindbdolphindbdolphindbdophindbdophindbdophindbdophindbdophindbdophindbpromeheuspromeheuspromeheuspromeheuspromeheuspromeheuspromeheuspromeheuspromeheuspromeheuspromeheusrulesgroupsnamealerulesalertforexprforlabelsseverityannotationssummarydescriptionjobinstancevaluesgeneratorURLrunbookurlreceiverreceiversnameemailconfigstofromsmarthostauthusernameauthpasswordauthidentityrequiretlsreceiversnamewechatconfigscorpidapisecretagentidtopartywebhookconfigsurlsendresolvedglobalresolveroutetimerepeatintervalroutegroupbyreceivergroupwaitgroupintervalrepeatintervalroutesmatchreceivercontinueinhibitrulestargetmatchsourcematchtargetmatchreequaloptsourcematchreequaloptalertmanageryamldockerdockercomposepromeusalertmanagergrafanadockerdockercomposepromeusalertmanagergrafanapromeusalertmanagergrafanapromeusalertmanagergrafanadockerdockercomposepromeusalertmanagergrafanadockerdockercomposepromeusalertmanagergrafanadockerdockercomposepromeusalertmanagergrafanadockerdockercomposepromeusalertmanagergrafanaversionservicespromeusimagecontainernamecommandconfig.filevolumesportsnetworksrdependsonrestartunless-stoppeddependsonrestartunless-stoppeddependsonrestartunless-stoppeddependsonrestartunless-stoppedvolumesnetworksdriverdocker-composeup-dhttplocalhostgraphhttplocalhostloginadminadminDataSourcesAddPromeusSaveDashboardsImportIDIDIDIDIDIDIDIDIDIDIDIDNetdataNetdataNetdataNetdataNetdataNetdataNetdataNetdataNetdataNetdatastreamstreamstreamstreamstreamstreamAPIKeypromeusremotewriteurlbasicauthusernamepasswordqueueconfigcapacitymaxsamplessentbatchsenddeadlineminshardminbackoffmaxbackoffremotewriteurlbasicauthusernamepasswordqueueconfigcapacitymaxsamplessentbatchsenddeadlineminshardminbackoffmaxbackoffnetdataconfnameserversenabledmemorymoderatelocalhostzeroconfigenabledbindtomemorymoderatelocalhostzeroconfigenabledbindtomemorymoderatelocalhostzeroconfigenabledbindtomemorymoderatelocalhostzeroconfigenabledbindtomemorymoderatelocalhostzeroconfigenabledbindtomemorymoderatelocalhostzeroconfigenabledbindtodashboardsjsongrafanaprovisioningdashboardsyamlapiversionprovidersnamedefaulttypeoptionspathupdateIntervalSecondsgrafanaprovisioningdatasourcesyamlapiversiondatasourcesnamedefaulttypeaccessurleditableisDefaulttrueJSONJSONJSONJSONJSONJSONYAMLYAMLYAMLYAMLYAMLYAMLYAMLYAMLYAMLYAMLYAMLEmailWeChat/DingTalkWebhookPhone/SMSZabbixServerZabbixAgentZabbixProxyZabbixServerZabbixAgentZabbixProxyTemplatesHostGroupsItemsTriggersActionsMediaTypesScriptsUserParametersUserParametersUserParametersUserParametersUserParametersUserParametersUserParametersUserParametersUserParametersUserParametersUserParametersechoechoechoechoechoechodfdudftopiostatvmstatsarnetstatechoechodfdudftopiostatvmstatsarnetstatechoechodfdudftopiostatvmstatsarnetstatechoechodfdudftopiostatvmstatsarnetstatechoechodfdudftopiostatvmstatsarnetstatechoechodfdudftopiostatvmstatsarnetstatechoechodfdudftopiostatvmstatsarnetstatesystemctljournalctltophtopvmstatiostatsarpidstatiotopiftopnloadnethogsdsatfreecat/proc/meminfonmonnmongrafanagrafanagrafanagrafanahttpsgithubcompromeusnodeexporternodeexporternodeexporternodeexporternodeexporternodeexporternodeexporternodeexporternodeexporternode_exportersystemdsystemdsystemdsystemdsystemdsystemdsystemdddddddddddddockerrunddetachpiddnetworkhostpidhostvolumerootfsrovolumeprocrwvolumesysrwvolumeetcrovolumeetcrovolumeroorootfsroprocrwsysrwetcroetcrodetachpiddnetworkhostpidhostvolumerootfsrovolumeprocrwvolumesysrwvolumeetcrovolumeetcrodetachpiddnetworkhostpidhostvolumerootfsrovolumeprocrwvolumesysrwvolumeetcrovolumeetcrodetachpiddnetworkhostpidhostvolumerootfsrovolumeprocrwvolumesysrwvolumeetcrovolumeetcrodetachpiddnetworkhostpidhostvolumerootfsrovolumeprocrwvolumesysrwvolumeetcrovolumeetcrodetachpiddnetworkhostpidhostvolumerootfsrovolumeprocrwvolumesysrwvolumeetcrovolumeetcrodetachpiddnetworkhostpidhostvolumerootfsrovolumeprocrwvolumesysrwvolumeetcrovolumeetcrodetachpiddnetworkhostpidhostvolumerootfsrovolumeprocrwvolumesysrwvolumeetcrovaluevaluevaluevaluevaluevaluevaluevaluevaluevaluenodenodenodenodenodenodenodenodenodenodenodenodeserverclustermetricsserverclustermetricsserverclustermetricsserverclustermetricsserverclustermetricsserverclustermetricsserverclustermetricscontrollernodemetricsmetapartitionmetadatatransactionqueryenginecompactionreplicationbackupmonitoringmonitoringmonitoringmonitoringmonitoringmonitoringsqlsqlsqlsqlsqlsqlsqlsqlsqlsqlexplainanalyzeshowjobsshowjobsshowjobsshowjobsshowjobsexplainanalyzeshowjobsexplainanalyzeshowjobsselectcountfromsystem.monitoringwhereeventTime娱乐weensqlselectcountfromsystem.monitoringwhereeventTime娱乐weenselectcountfromsystem.monitoringwhereeventTime娱乐weenselectcountfromsystem.monitoringwhereeventTime娱乐weenselectcountfromsystem.monitoringwhereeventTime娱乐weensqlsqlexplainanalyzeshowjobsselectcountfromsystem.monitoringwhereeventTime娱乐weenexplainanalyzeshowjobsselectcountfromsystem.monitoringwhereeventTime娱乐weenexplainanalyzeshowjobsselectcountfromsystem.monitoringwhereeventTime娱乐weenexplainanalyzeshowjobsselectcountfromsystem.monitoringwhereeventTime娱乐weenexplainanalyzeshowjobsselectcountfromsystem.monitoringwhereeventTime娱乐weenexplainanalyzeshowjobssecuritycentermanagealarmcreatealarmEmailPluginSMTPServerPortUsernamePasswordReceiverSecurityCenterAlarmInstanceEmailPluginSMTPServerPortUsernamePasswordReceiverSecurityCenterAlarmInstanceSecurityCenterAlarmInstanceSecurityCenterAlarmInstanceSecurityCenterAlarmInstanceSecurityCenterAlarmInstanceSecurityCenterAlarmInstanceSecurityCenterAlarmInstanceSecurityCenterAlarmInstanceSecurityCenterAlarmInstanceparseparseparseparseparseparseparsecurl-XPOST-HContent-Typeapplication/json-dcontentcurl-XPOST-HContent-Typeapplication/json-dcontentcurl-XPOST-HContent-Typeapplication/json-dcontentcurl-XPOST-HContent-Typeapplication/json-dcontentcurl-XPOST-HContent-Typeapplication/json-dcontentcurl-XPOST-HContent-Typeapplication/json-dcontentcurl-XPOST-HContent-Typeapplication/json-dcontentcustomshellscriptcustomshellscriptcustomshellscriptcustomshellscriptcustomshellscriptcustomshellscriptcustomshellscriptshelldsdolschainschedulerschedulerschedulerschedulerschedulerschedulerschedulerschedulerworkflowworkflowworkflowworkflowtasktasktasktaskinstancerunrunrunrunrunningfailedsuccesssuccesssuccesssuccesssuccessfailureretryretryretryretryretryretryretryretrywaitingtimingtimingtimingtimingtimingtimingtimingtaskinstancerunningfailedwaitingtaskinstancerunningfailedwaitingtaskinstancerunningfailedwaitingtaskinstancerunningfailedwaitingtaskinstancerunningfailedwaitingingingingingingingingingingingingingingingingingingingingingingingingingingingerorsuccessfailureretryrunningwaitingsuccessfailureretryrunningwaitingsuccessfailureretryrunningwaitingsuccessfailureretryrunningwaitingsuccessfailureretryrunningwaitingsuccessfailureretryrunningwaitingsuccessfailureretryrunningwaitingshellshellshellshellaaaabbbcccccceeeeeeffffffffggggggghhhhhhhiiiiiiiijjjjjjjkkkkkkklllllllmmmmmmmnnnnnnnoooooooopppppppqqqqqqqrrrrrrrssssssstttttttuuuuuuuvvvvvvvwwwwwwwxxxxxxxxyyyyyyyzzzzzzzAAAAAAABBBBBB娱乐CCCCCCDDDDDDDEEEEEEEFFFFFFFFGGGGGGGHHHHHHHIIIIIIIJJJJJJJKKKKKKKLLLLLLLMMMMMMMNNNNNNNOOOOOOOPPPPPPPQQQQQQQRRRRRRRSSSSSSSTTTTTTTUUUUUUUVVVVVVVWWWWWWWXXXXXXXYYYYYYYZZZZZZZaaaaaaaabbbbbbbcccccccddddddd

💡 常用方法建议： 🎯 阈值分级： 🔔 P Critical ：立即电话短信叫醒人；⚠️ P Warning ：工作时间钉钉企微群通知；📊 P Info ：仅记录日志/Grafana Annotation。⏳ 持续时间 for 防抖动： expressionforlabelsseverityannotationssummarydescriptiongeneratorURLrunbookurl 表达式持续时间标签注解摘要描述生成器URL运行手册链接 📉 动态阈值： ratepredictlinearpredictlinearpredictlinearpredictlinearpredictlinearpredictlinearpredictlinearpredictlinearratepredictlinearpredictlinearratepredictlinearratepredictlinearratepredictlinearratepredictlinearratepredictlineratepredictlineratepredictlineratepredictlineratepredictlineratepredictrateraterateraterateraterateraterateraterecordrecordrecordrecordrecordrecordrulegroupsnamrulesrecordrecordrecordrulegroupsnamrulesrulegroupsnamrulesrulegroupsnamrulesrulegroupsnamrulesrecordingrulesgroupsnamesintervalrulesrecordexprlabelsrecordexprlabelsrecordexprlabelsrecordexprlabelsrecordexprlabelsrecordexprlabelsrecordingrulesgroupsnamesintervalrulesrecordexprlabelsrecordingrulesgroupsnamesintervalrulesrecordexprlabelsrecordingrulesgroupsnamesintervalrulesrecordexprlabelsrecordingrulesgroupsnamesintervalrulesrecordexprlabelspromtoolcheckrulestooltooltooltooltooltooltooltooltooltooltoolsshsshsshsshsshsshsshknownhostsknownhostsknownhostsknownhostsknownhostsknownhostsknownhostsknownhostsknownhostskeyekeykeykeykeykeykeykeypubpubpubpubpubpubpubprivprivprivprivprivprivprivauthorizedkeysauthorizedkeysauthorizedkeysauthorizedkeysauthorizedkeysauthorized_keysfirewallcmdfirewallcmdfirewallcmdfirewallcmdfirewallcmdfirewallcmdfirewallcmdpermanentaddportpermanentaddportpermanentaddportpermanentaddportpermanentaddportpermanentaddportreloadreloadreloadreloadreloadselinuxselinuxselinuxselinuxselinuxselinuxselinuxenforcingenforcingenforcingenforcingenforcingenforcingpermissivepermissivepermissivepermissivepermissivepermissivedisableddisableddisableddisableddisabledenabledenabledenabledenabledenabledtimetimetimetimetimetimenptimenptimenptimenptimenptimesynconcesynconcesynconcesynconcesynconcesynconcechronychronychronychronychronyconfconfconfconfconfpoolpoolpoolpoolpooliburstiburstiburstiburstiburstiburstmakestepmakestepmakestepmakestepmakestepsystemsystemsystemsytemdtimesyncdtimesyncdtimesyncdtimesyncdtimesyncdstatusstatusstatusstatusstatusactiveactiveactiveactiveactiveservicesservicesservicesservicesservicesntpdntpdntpdntpdntpdchronydchronydchronydchronydchronyd

通过以上程序化搭建——从 OS 底座 的 Node Exporter + Netdata 快速洞察。到 Dolphin 内核 的 Promeus Exporter + SQL 深度巡检，再到 Alertmanager + Zabbix/DolphinScheduler 的多渠道触达与分级降噪——你将彻底告别“事后诸葛亮”，真正实现对 CentOS Dolphin 集群的全生命周期健康管控。现在就动手部署第一个 Exporter吧！