Detect hang by tensor util

[majieyue]

What changes were proposed in this pull request?

upgrade metric module to provide metric trace api
add metric collector thread in diagnosis manager
add --detect-hang-by-xpu-util paramters to restart job when xpu util drop to zero

Why are the changes needed?

Use basic tensor util(GPU) or NPU util(NPU) to detect job hang

Does this PR introduce any user-facing change?

YES, add --xpu_type, --hang_detect and --hang_downtime in job args

How was this patch tested?

UT

[codecov]

Codecov Report

Attention: Patch coverage is 85.85859% with 70 lines in your changes missing coverage. Please review.

Project coverage is 81.62%. Comparing base (03c965f) to head (a47a4e5).
Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
...rover/python/master/diagnosis/diagnosis_manager.py	75.00%	22 Missing ⚠️
dlrover/python/common/metric/monitor.py	66.12%	21 Missing ⚠️
dlrover/python/common/metric/context.py	79.03%	13 Missing ⚠️
dlrover/python/master/main.py	0.00%	8 Missing ⚠️
dlrover/python/common/metric/metric.py	90.00%	2 Missing ⚠️
dlrover/python/master/dist_master.py	60.00%	2 Missing ⚠️
dlrover/python/elastic_agent/torch/training.py	50.00%	1 Missing ⚠️
dlrover/python/master/node/dist_job_manager.py	75.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1448      +/-   ##
==========================================
+ Coverage   81.53%   81.62%   +0.09%     
==========================================
  Files         240      240              
  Lines       23592    24045     +453     
==========================================
+ Hits        19235    19626     +391     
- Misses       4357     4419      +62     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[BalaBalaYi]

LGTM