案例分析一次C4506交换机CPU利用率过高的处理过程

案例分析：一次C4506交换机CPU利用率过高的处理过程刚处理过一次C4506交换机CPU利用率过高（1）通过交换机showproccpu的命令输出的信息可以看到：在5秒、1分钟、5分钟内Cat4kMgmtLoPri进程的CPU占用率分别为91.69%92.18%92.23%显然是这个进程消耗了大量的CPU运行资源，从而导致交换机没有新的资源处理新的请求。处理器运行两个进程，Cat4kMgmtHiPri和Cat4kMgmtLoPri，这两个进程同其他进程一样要占用cpu时间。当Cat4k平台上某项进程占用cpu超出了应分配的cpu时间，Cat4kMgmtLoPri会接管这项进程，使其他进程能够得到cpu时间。同样当某个进程占用cpu时间没有超过规定的cpu利用率，Cat4kMgmtHiPri会接管这个进程。而Cat4kMgmtLoPri进程CPU占用率超过90%，我们判断一定是某一个进程大大超过了CPU应分配的时间，而Cat4kMgmtLoPri进程试图接管这一进程，而导致了Cat4kMgmtLoPri进程的CPU占用时间超过了90%。根据这一思路只要我们找出这个进程，将其关闭或找出这个进程的作用，就可以解决这个问题并能够找出CPU占用率过高的原因。showproccpuCPUutilizationforfiveseconds:99%/0%;oneminute:99%;fiveminutes:99%PIDRuntime(ms)InvokeduSecs5Sec1Min5MinTTYProcess101300.00%0.00%0.00%0ChunkManager255221883250.00%0.00%0.00%0LoadMeter316013112210.00%0.00%0.00%0SpanTreeHelper40100.00%0.00%0.00%0DeferredEvents51039241473270540.00%0.06%0.05%0Checkheaps64220000.00%0.00%0.00%0PoolManager70200.00%0.00%0.00%0Timers80200.00%0.00%0.00%0SerialBackgroun90100.00%0.00%0.00%0AAA_SERVER_DEADT100200.00%0.00%0.00%0AAAhigh-capacit110100.00%0.00%0.00%0PolicyManager1216182580.00%0.00%0.00%0IPCDynamicCach130100.00%0.00%0.00%0IPCZoneManager1464410913550.00%0.00%0.00%0IPCPeriodicTim1541210913630.00%0.00%0.00%0IPCDeferredPor160100.00%0.00%0.00%0IPCSeatManager170100.00%0.00%0.00%0IFSAgentManage181624845140053160.23%0.21%0.22%0ARPInput19681161810.00%0.00%0.00%0EntityMIBAPI200100.00%0.00%0.00%0SERIALA'detect2185610913670.00%0.00%0.00%0DynamicARPInsp22206027333750.00%0.00%0.00%0HCCounterTimer230100.00%0.00%0.00%0CriticalBkgnd24171691507180.00%0.00%0.00%0NetBackground250500.00%0.00%0.00%0Logger2671610911760.00%0.00%0.00%0TTYBackground272072111013180.00%0.00%0.00%0Per-SecondJobs28382921947196670.00%0.04%0.00%0Per-minuteJobs29285850888073603244.63%3.81%3.72%0Cat4kMgmtHiPri308437156467467994125091.69%92.18%92.23%0Cat4kMgmtLoPri312604935850.00%0.00%0.00%0GaliosReschedul320200.00%0.00%0.00%0IOSACLHelper330200.00%0.00%0.00%0NAMManager341522760000.00%0.00%0.00%0rftask3512428405853300.00%0.00%0.00%0NetInput3611620230225040.07%0.03%0.00%0Computeloadavg3708400.00%0.00%0.00%0BACKCHECK380100.00%0.00%0.00%0chkptmessageha390200.00%0.00%0.00%0cpf_process_msg_400100.00%0.00%0.00%0cpf_process_ipcQ4101400.00%0.00%0.00%0AggMgrProcess420100.00%0.00%0.00%0SFF84724301500.00%0.00%0.00%0Collectionproce440300.00%0.00%0.00%0CEFswitchingba450200.00%0.00%0.00%0AAADictionaryR460200.00%0.00%0.00%0AAAServer470100.00%0.00%0.00%0AAAACCTProc480100.00%0.00%0.00%0ACCTPeriodicPr4968882413997074920.71%0.62%0.62%0SpanningTree50439655767780.00%0.01%0.00%0DTPProtocol51401094630.00%0.00%0.00%0Ethchnl523744109502340.00%0.01%0.00%0UDLD5312934120.00%0.00%0.00%0DHCPSnooping54201825100.00%0.00%0.00%0Port-Security55252170049279115111.59%1.75%1.77%0IPInput560100.00%0.00%0.00%0ICMPeventhandl57919083791024240.07%0.14%0.11%0CDPProtocol5816221720.00%0.02%0.00%0Exec611482534580.00%0.00%0.00%0CEFbackgroundp620200.00%0.00%0.00%0XDRmcast630100.00%0.00%0.00%0IPCLCMessageH640100.00%0.00%0.00%0XDRRPPingBack65892580.00%0.00%0.00%0XDRRPbackgroun660100.00%0.00%0.00%0XDRRPTestBack670100.00%0.00%0.00%0IPIRDP68880641599285500.07%0.07%0.07%0CEF:IPv4proces691843651110.00%0.00%0.00%0ADJbackground70483671300.00%0.00%0.00%0L2MM711201447820.00%0.00%0.00%0MRD72106811946890.00%0.00%0.00%0IGMPSN730100.00%0.00%0.00%0IGMPSN-HA740100.00%0.00%0.00%0SocketTimers750200.00%0.00%0.00%0L2TRACESERVER7628816798170.00%0.00%0.00%0TCPTimer7740517840.00%0.00%0.00%0TCPProtocols78101256411790.00%0.00%0.00%0HTTPCORE790100.00%0.00%0.00%0CHKPTEXAMPLE800100.00%0.00%0.00%0CHKPTDevTest810200.00%0.00%0.00%0ATIP_UDP_TSK820100.00%0.00%0.00%0DHCPSnoopingHA830100.00%0.00%0.00%0ProbeInput840100.00%0.00%0.00%0RARPInput855380338081590.00%0.00%0.00%0DHCPDReceive86113220795440.00%0.00%0.00%0IPBackground8730018561610.00%0.00%0.00%0IPRIBUpdate880100.00%0.00%0.00%0COPS89161371110.00%0.00%0.00%0ClusterL290761094660.00%0.00%0.00%0ClusterRARP910200.00%0.00%0.00%0LOCALAAA920200.00%0.00%0.00%0AAACachedServe930200.00%0.00%0.00%0TPLUS940300.00%0.00%0.00%0RADIUSTESTCMD950200.00%0.00%0.00%0AAASENDSTOPEV9611256754707140.15%0.04%0.01%0PMCallback9738030391250.00%0.00%0.00%0VLANManager98891380.00%0.00%0.00%0DHCPDTimer994162500.00%0.00%0.00%0VTPTrapProcess1000200.00%0.00%0.00%0DHCPSecurityHe1010100.00%0.00%0.00%0DiagCard1/-11020100.00%0.00%0.00%0DiagCard2/-11030100.00%0.00%0.00%0DiagCard3/-11040100.00%0.00%0.00%0DiagCard4/-11054124768650.00%0.00%0.00%0SyslogTraps1060200.00%0.00%0.00%0VTPMIBEDITBUFF1070300.00%0.00%0.00%0SPANswitch1080200.00%0.00%0.00%0SNMPTimers1090200.00%0.00%0.00%0IPSNMP1100100.00%0.00%0.00%0PDUDISPATCHER1110100.00%0.00%0.00%0SNMPENGINE1120100.00%0.00%0.00%0SNMPConfCopyPro113804916320.00%0.00%0.00%0SNMPTraps1141344111625120.00%0.00%0.00%0NTP11532430976100.00%0.00%0.00%0DHCPDDatabase1163565465360.00%0.00%0.00%0Systempolling通过showplatformhealth命令进一步更为详细的检查Catalyst4506交换机的进程我们发现K2CpuManReview进程有异常，如下：showplahea%CPU%CPURunTimeMaxPriorityAverage%CPUTotalTargetActualTargetActualFgBg5SecMinHourCPULj-poll1.000.0121441005000000:26GalChassisVp-review3.000.171028100500