做生存曲线的网站,网络经营许可证查询,网站建设模板个人,用html做静态网站UniADhttps://github.com/OpenDriveLab/UniAD是面向行车规划集感知(目标检测与跟踪)、建图(不是像SLAM那样对环境重建的建图#xff0c;而是实时全景分割图像里的道路、隔离带等行车需关注的相关物体)、和轨迹规划和占用预测等多任务模块于一体的统一大模型。官网上的安装说明…UniADhttps://github.com/OpenDriveLab/UniAD是面向行车规划集感知(目标检测与跟踪)、建图(不是像SLAM那样对环境重建的建图而是实时全景分割图像里的道路、隔离带等行车需关注的相关物体)、和轨迹规划和占用预测等多任务模块于一体的统一大模型。官网上的安装说明是按作者使用的较低版本的CUDA11.1.1和pytorch1.9.1来的对应的mmcv也是较低版本的1.4版我们工作服务器上的nvidia ngc docker环境里使用的这些支持工具软件早已是较高版本的于是想在我们自己的环境里把UniAD跑起来安装过程中遇到一些坑最终都一一解决了也实测过了UniAD完全可以正常跑在CUDA11.6pytorch1.12.0mmcv1.6mmseg0.24.0mmdet2.24mmdet3d1.0.0rc4组成的环境下。
安装和解决问题的步骤如下:
1.拉取使用CUDA11.6的NVIDIA NGC docker镜像并创建容器作为UniAD的运行环境
2.安装pytorch和torchvision:
pip install torch1.12.0cu116 torchvision0.13.0cu116 torchaudio0.12.0 --extra-index-url https://download.pytorch.org/whl/cu116
3.检查CUDA_HOME环境变量是否已设置如果没有则设置一下
export CUDA_HOME/usr/local/cuda
4.受CUDA和pytorch版本限制mmcv需要安装比1.4高版本的1.6.0(关于openmmlab的mm序列框架包的安装版本对应关系参见openMMLab的mmcv和mmdet、mmdet3d、mmseg版本对应关系-CSDN博客: ) pip install mmcv-full1.6.0 -f https://download.openmmlab.com/mmcv/dist/cu116/torch1.12.0/index.html
按照下面的对应关系 分别安装mmdet2.24.0和mmseg0.24.0
pip install mmdet2.24.0 pip install mmsegmentation0.24.0
下载mmdetection3d源码然后切换到v1.0.0rc4版:
git clone https://github.com/open-mmlab/mmdetection3d.git cd mmdetection3d
git checkout v1.0.0rc4
安装支持包并从源码编译和安装mmdet3d:
pip install scipy1.7.3 pip install scikit-image0.20.0 #将requirements/runtime.txt里修改一下numba的版本: numba0.53.1 #numba0.53.0 pip install -v -e . 安装好了支持环境然后下载和安装UniAD:
git clone https://github.com/OpenDriveLab/UniAD.git cd UniAD
#修改一下requirements.txt里的numpy版本然后安装相关支持包: #numpy1.20.0 numpy1.22.0 pip install -r requirements.txt
#下载相关预训练权重文件 mkdir ckpts cd ckpts wget https://github.com/zhiqi-li/storage/releases/download/v1.0/bevformer_r101_dcn_24ep.pth wget https://github.com/OpenDriveLab/UniAD/releases/download/v1.0/uniad_base_track_map.pth wget https://github.com/OpenDriveLab/UniAD/releases/download/v1.0.1/uniad_base_e2e.pth
按照https://github.com/OpenDriveLab/UniAD/blob/main/docs/DATA_PREP.md说明下载和展开整理NuScenes数据集
下载data infos文件:
cd UniAD/data
mkdir infos cd infos
wget https://github.com/OpenDriveLab/UniAD/releases/download/v1.0/nuscenes_infos_temporal_train.pkl # train_infos
wget https://github.com/OpenDriveLab/UniAD/releases/download/v1.0/nuscenes_infos_temporal_val.pkl # val_infos
假如NuScenes数据集和data infos文件已经下载好并解压存放到 ./data/下了并按照下面的结构存放的:
UniAD
├── projects/
├── tools/
├── ckpts/
│ ├── bevformer_r101_dcn_24ep.pth
│ ├── uniad_base_track_map.pth
| ├── uniad_base_e2e.pth
├── data/
│ ├── nuscenes/
│ │ ├── can_bus/
│ │ ├── maps/
│ │ │ ├──36092f0b03a857c6a3403e25b4b7aab3.png
│ │ │ ├──37819e65e09e5547b8a3ceaefba56bb2.png
│ │ │ ├──53992ee3023e5494b90c316c183be829.png
│ │ │ ├──93406b464a165eaba6d9de76ca09f5da.png
│ │ │ ├──basemap
│ │ │ ├──expansion
│ │ │ ├──prediction
│ │ ├── samples/
│ │ ├── sweeps/
│ │ ├── v1.0-test/
│ │ ├── v1.0-trainval/
│ ├── infos/
│ │ ├── nuscenes_infos_temporal_train.pkl
│ │ ├── nuscenes_infos_temporal_val.pkl
│ ├── others/
│ │ ├── motion_anchor_infos_mode6.pkl
注意map(v1.3) extensions压缩包下载后展开的三个目录basemap、expansion、prediction需要放在maps目录下而不是和samples、sweeps等目录平级NuScenes的train所有数据压缩包展开后samples的最底层的每个子目录下都是34149张图片sweeps里的子母录下的图片数量则是不等的例如163881、164274、164166、161453、160856、164266...等把没有标注的test数据的压缩包在nuscenes目录下展开后其里面samples和sweeps目录里子目录下的图片会自动拷贝到nuscenes/samples和nuscenes/sweeps下的对应子目录里去再次统计会看到samples下的每个子目录里的图片数量变成了40157而sweeps下的子目录里的图片数量则变成了193153、189171、189905、193082、193168、192699... 执行
./tools/uniad_dist_eval.sh ./projects/configs/stage1_track_map/base_track_map.py ./ckpts/uniad_base_track_map.pth 8
运行一下试试看最后一个参数是GPU个数我的工作环境和作者的工作环境一样都是8张A100卡所以照着做如果卡少修改这个参数例如使用1也是可以跑的只是比较慢。 第一次运行上面命令可能会遇到下面的问题
1. partially initialized module cv2 has no attribute _registerMatType (most likely due to a circular import)
这是因为环境里的opencv-python版本太高了版本不兼容引起的我的是4.8.1.78查了一下网上需要降到4.5执行下面的命令重新安装opencv-python4.5即可 pip install opencv-python4.5.4.58
2. ImportError: libGL.so.1: cannot open shared object file: No such file or directory
安装libgl即可: sudo apt-get update sudo apt-get install libgl1
3. AssertionError: MMCV1.6.0 is used but incompatible. Please install mmcv(1, 3, 13, 0, 0, 0), (1, 5, 0, 0, 0, 0)
Traceback (most recent call last):File tools/create_data.py, line 4, in modulefrom data_converter import uniad_nuscenes_converter as nuscenes_converterFile /workspace/workspace_fychen/UniAD/tools/data_converter/uniad_nuscenes_converter.py, line 13, in modulefrom mmdet3d.core.bbox.box_np_ops import points_cam2imgFile /workspace/workspace_fychen/mmdetection3d/mmdet3d/__init__.py, line 5, in moduleimport mmsegFile /opt/conda/lib/python3.8/site-packages/mmseg/__init__.py, line 58, in moduleassert (mmcv_min_version mmcv_version mmcv_max_version), \
AssertionError: MMCV1.6.0 is used but incompatible. Please install mmcv(1, 3, 13, 0, 0, 0), (1, 5, 0, 0, 0, 0).
这错误是python3.8/site-packages/mmseg/__init__.py抛出来的说明mmseg和mmcv1.6.0版本不兼容它要求安装mmcv的1.3-1.5版说明mmseg自身版本低了原因是开始安装的mmsegmenation版本低了改安装mmseg0.24.0即可。其它功能框架包遇到版本问题做类似处理。
4.KeyError: DiceCost is already registered in Match Cost
Traceback (most recent call last):File ./tools/test.py, line 16, in modulefrom projects.mmdet3d_plugin.datasets.builder import build_dataloaderFile /workspace/workspace_fychen/UniAD/projects/mmdet3d_plugin/__init__.py, line 3, in modulefrom .core.bbox.match_costs import BBox3DL1Cost, DiceCostFile /workspace/workspace_fychen/UniAD/projects/mmdet3d_plugin/core/bbox/match_costs/__init__.py, line 2, in modulefrom .match_cost import BBox3DL1Cost, DiceCostFile /workspace/workspace_fychen/UniAD/projects/mmdet3d_plugin/core/bbox/match_costs/match_cost.py, line 32, in moduleTraceback (most recent call last):
class DiceCost(object):File /opt/conda/lib/python3.8/site-packages/mmcv/utils/registry.py, line 337, in _registerFile ./tools/test.py, line 16, in modulefrom projects.mmdet3d_plugin.datasets.builder import build_dataloaderFile /workspace/workspace_fychen/UniAD/projects/mmdet3d_plugin/__init__.py, line 3, in modulefrom .core.bbox.match_costs import BBox3DL1Cost, DiceCostFile /workspace/workspace_fychen/UniAD/projects/mmdet3d_plugin/core/bbox/match_costs/__init__.py, line 2, in modulefrom .match_cost import BBox3DL1Cost, DiceCostFile /workspace/workspace_fychen/UniAD/projects/mmdet3d_plugin/core/bbox/match_costs/match_cost.py, line 32, in moduleclass DiceCost(object):File /opt/conda/lib/python3.8/site-packages/mmcv/utils/registry.py, line 337, in _registerself._register_module(modulemodule, module_namename, forceforce)self._register_module(modulemodule, module_namename, forceforce)File /opt/conda/lib/python3.8/site-packages/mmcv/utils/misc.py, line 340, in new_funcFile /opt/conda/lib/python3.8/site-packages/mmcv/utils/misc.py, line 340, in new_funcoutput old_func(*args, **kwargs)File /opt/conda/lib/python3.8/site-packages/mmcv/utils/registry.py, line 272, in _register_moduleraise KeyError(f{name} is already registered
KeyError: DiceCost is already registered in Match Cost
这种类注册重复了的问题是因为UniAD的mmdet3d_plugin和我安装的mmdetection的文件python3.8/site-packages/mmdet/core/bbox/match_costs/match_cost.py里有同名的DiceCost类(UniAD作者使用的mmdetection版本较低应该没有这个问题)读mmcv里面python3.8/site-packages/mmcv/utils/registry.py的注册代码可以知道这个问题可以设置参数forceTrue来解决 deprecated_api_warning(name_dictdict(module_classmodule))def _register_module(self, module, module_nameNone, forceFalse):if not inspect.isclass(module) and not inspect.isfunction(module):raise TypeError(module must be a class or a function, fbut got {type(module)})if module_name is None:module_name module.__name__if isinstance(module_name, str):module_name [module_name]for name in module_name:if not force and name in self._module_dict:raise KeyError(f{name} is already registered fin {self.name})self._module_dict[name] module
为了保证UniAD代码能正确运行允许UniAD的DiceCost类强制注册即可也就是修改UniAD/projects/mmdet3d_plugin/core/bbox/match_costs/match_cost.py里DiceCost类的装饰器语句增加forceTrue参数
MATCH_COST.register_module(forceTrue)
class DiceCost(object):5.TypeError: cannot pickle dict_keys object
File ./tools/test.py, line 261, in modulemain()File ./tools/test.py, line 231, in mainoutputs custom_multi_gpu_test(model, data_loader, args.tmpdir,File /workspace/workspace_fychen/UniAD/projects/mmdet3d_plugin/uniad/apis/test.py, line 88, in custom_multi_gpu_testfor i, data in enumerate(data_loader):File /opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py, line 438, in __iter__return self._get_iterator()File /opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py, line 384, in _get_iteratorreturn _MultiProcessingDataLoaderIter(self)File /opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py, line 1048, in __init__w.start()File /opt/conda/lib/python3.8/multiprocessing/process.py, line 121, in startself._popen self._Popen(self)File /opt/conda/lib/python3.8/multiprocessing/context.py, line 224, in _Popenreturn _default_context.get_context().Process._Popen(process_obj)File /opt/conda/lib/python3.8/multiprocessing/context.py, line 284, in _Popenreturn Popen(process_obj)File /opt/conda/lib/python3.8/multiprocessing/popen_spawn_posix.py, line 32, in __init__super().__init__(process_obj)File /opt/conda/lib/python3.8/multiprocessing/popen_fork.py, line 19, in __init__self._launch(process_obj)File /opt/conda/lib/python3.8/multiprocessing/popen_spawn_posix.py, line 47, in _launchreduction.dump(process_obj, fp)File /opt/conda/lib/python3.8/multiprocessing/reduction.py, line 60, in dumpForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle dict_keys object
解决办法参见 如何定位TypeError: cannot pickle dict_keys object错误原因及解决NuScenes数据集在多进程并发训练或测试时出现的这个错误-CSDN博客
6.protobuf报错 TypeError: Descriptors cannot not be created directly
Traceback (most recent call last):File ./tools/test.py, line 16, in modulefrom projects.mmdet3d_plugin.datasets.builder import build_dataloaderFile /workspace/workspace_fychen/UniAD/projects/mmdet3d_plugin/__init__.py, line 5, in modulefrom .datasets.pipelines import (File /workspace/workspace_fychen/UniAD/projects/mmdet3d_plugin/datasets/pipelines/__init__.py, line 6, in modulefrom .occflow_label import GenerateOccFlowLabelsFile /workspace/workspace_fychen/UniAD/projects/mmdet3d_plugin/datasets/pipelines/occflow_label.py, line 5, in modulefrom projects.mmdet3d_plugin.uniad.dense_heads.occ_head_plugin import calculate_birds_eye_view_parametersFile /workspace/workspace_fychen/UniAD/projects/mmdet3d_plugin/uniad/__init__.py, line 2, in modulefrom .dense_heads import *File /workspace/workspace_fychen/UniAD/projects/mmdet3d_plugin/uniad/dense_heads/__init__.py, line 4, in modulefrom .occ_head import OccHeadFile /workspace/workspace_fychen/UniAD/projects/mmdet3d_plugin/uniad/dense_heads/occ_head.py, line 16, in modulefrom .occ_head_plugin import MLP, BevFeatureSlicer, SimpleConv2d, CVT_Decoder, Bottleneck, UpsamplingAdd, \File /workspace/workspace_fychen/UniAD/projects/mmdet3d_plugin/uniad/dense_heads/occ_head_plugin/__init__.py, line 1, in modulefrom .metrics import *File /workspace/workspace_fychen/UniAD/projects/mmdet3d_plugin/uniad/dense_heads/occ_head_plugin/metrics.py, line 10, in modulefrom pytorch_lightning.metrics.metric import MetricFile /opt/conda/lib/python3.8/site-packages/pytorch_lightning/__init__.py, line 29, in modulefrom pytorch_lightning.callbacks import Callback # noqa: E402File /opt/conda/lib/python3.8/site-packages/pytorch_lightning/callbacks/__init__.py, line 25, in modulefrom pytorch_lightning.callbacks.swa import StochasticWeightAveragingFile /opt/conda/lib/python3.8/site-packages/pytorch_lightning/callbacks/swa.py, line 26, in modulefrom pytorch_lightning.trainer.optimizers import _get_default_scheduler_configFile /opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/__init__.py, line 18, in modulefrom pytorch_lightning.trainer.trainer import TrainerFile /opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py, line 30, in modulefrom pytorch_lightning.loggers import LightningLoggerBaseFile /opt/conda/lib/python3.8/site-packages/pytorch_lightning/loggers/__init__.py, line 18, in modulefrom pytorch_lightning.loggers.tensorboard import TensorBoardLoggerFile /opt/conda/lib/python3.8/site-packages/pytorch_lightning/loggers/tensorboard.py, line 25, in modulefrom torch.utils.tensorboard import SummaryWriterFile /opt/conda/lib/python3.8/site-packages/torch/utils/tensorboard/__init__.py, line 12, in modulefrom .writer import FileWriter, SummaryWriter # noqa: F401File /opt/conda/lib/python3.8/site-packages/torch/utils/tensorboard/writer.py, line 9, in modulefrom tensorboard.compat.proto.event_pb2 import SessionLogFile /opt/conda/lib/python3.8/site-packages/tensorboard/compat/proto/event_pb2.py, line 17, in modulefrom tensorboard.compat.proto import summary_pb2 as tensorboard_dot_compat_dot_proto_dot_summary__pb2File /opt/conda/lib/python3.8/site-packages/tensorboard/compat/proto/summary_pb2.py, line 17, in modulefrom tensorboard.compat.proto import tensor_pb2 as tensorboard_dot_compat_dot_proto_dot_tensor__pb2File /opt/conda/lib/python3.8/site-packages/tensorboard/compat/proto/tensor_pb2.py, line 16, in modulefrom tensorboard.compat.proto import resource_handle_pb2 as tensorboard_dot_compat_dot_proto_dot_resource__handle__pb2File /opt/conda/lib/python3.8/site-packages/tensorboard/compat/proto/resource_handle_pb2.py, line 16, in modulefrom tensorboard.compat.proto import tensor_shape_pb2 as tensorboard_dot_compat_dot_proto_dot_tensor__shape__pb2File /opt/conda/lib/python3.8/site-packages/tensorboard/compat/proto/tensor_shape_pb2.py, line 36, in module_descriptor.FieldDescriptor(File /opt/conda/lib/python3.8/site-packages/google/protobuf/descriptor.py, line 561, in __new___message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:1. Downgrade the protobuf package to 3.20.x or lower.2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATIONpython (but this will use pure-Python parsing and will be much slower).
我的protobuf版本4.24.4太高了降为3.20后就可以了 pip install protobuf3.20
7. TypeError: expected str, bytes or os.PathLike object, not _io.BufferedReader
Traceback (most recent call last):File /opt/conda/lib/python3.8/site-packages/mmcv/utils/registry.py, line 69, in build_from_cfgreturn obj_cls(**args)File /workspace/workspace_fychen/UniAD/projects/mmdet3d_plugin/datasets/nuscenes_e2e_dataset.py, line 78, in __init__super().__init__(*args, **kwargs)File /workspace/workspace_fychen/mmdetection3d/mmdet3d/datasets/nuscenes_dataset.py, line 131, in __init__super().__init__(File /workspace/workspace_fychen/mmdetection3d/mmdet3d/datasets/custom_3d.py, line 88, in __init__self.data_infos self.load_annotations(open(local_path, rb))File /workspace/workspace_fychen/UniAD/projects/mmdet3d_plugin/datasets/nuscenes_e2e_dataset.py, line 152, in load_annotationsdata pickle.loads(self.file_client.get(ann_file))File /opt/conda/lib/python3.8/site-packages/mmcv/fileio/file_client.py, line 1014, in getreturn self.client.get(filepath)File /opt/conda/lib/python3.8/site-packages/mmcv/fileio/file_client.py, line 535, in getwith open(filepath, rb) as f:
TypeError: expected str, bytes or os.PathLike object, not _io.BufferedReader
问题原因出现在高版本的mmdetection3d/mmdet3d/datasets/custom_3d.py里考虑了支持local_path读取文件传入load_annotations()的就是个io句柄了
def __init__(self,data_root,ann_file,pipelineNone,classesNone,modalityNone,box_type_3dLiDAR,filter_empty_gtTrue,test_modeFalse,file_client_argsdict(backenddisk)):super().__init__()self.data_root data_rootself.ann_file ann_fileself.test_mode test_modeself.modality modalityself.filter_empty_gt filter_empty_gtself.box_type_3d, self.box_mode_3d get_box_type(box_type_3d)self.CLASSES self.get_classes(classes)self.file_client mmcv.FileClient(**file_client_args)self.cat2id {name: i for i, name in enumerate(self.CLASSES)}# load annotationsif not hasattr(self.file_client, get_local_path):with self.file_client.get_local_path(self.ann_file) as local_path:self.data_infos self.load_annotations(open(local_path, rb))else:warnings.warn(The used MMCV version does not have get_local_path. fWe treat the {self.ann_file} as local paths and it might cause errors if the path is not a local path. Please use MMCV 1.3.16 if you meet errors.)self.data_infos self.load_annotations(self.ann_file)
根源是UniAD的UniAD/projects/mmdet3d_plugin/datasets/nuscenes_e2e_dataset.py里
在实现load_annotations()时默认是只支持使用ann_file是字符串类型所以这里强制修改一下mmdetection3d/mmdet3d/datasets/custom_3d.py改回使用self.data_infos self.load_annotations(self.ann_file)即可。
8. RuntimeError: DataLoader worker (pid 33959) is killed by signal: Killed
前面的7个问题都解决后如果NuScenes数据集是完整的且位置正确的话运行下面的命令应该都可以运行:
./tools/uniad_dist_eval.sh ./projects/configs/stage1_track_map/base_track_map.py ./ckpts/uniad_base_track_map.pth 8 ./tools/uniad_dist_eval.sh ./projects/configs/stage2_e2e/base_e2e.py ./ckpts/uniad_base_e2e.pth 8
但是可能会在循环读取数据时发生超时错误而导致dataloader所在进程被杀掉
Traceback (most recent call last):File /opt/conda/lib/python3.8/site-packages/torch/utils/data/dataloader.py, line 1134, in _try_get_datadata self._data_queue.get(timeouttimeout)File /opt/conda/lib/python3.8/multiprocessing/queues.py, line 107, in getif not self._poll(timeout):File /opt/conda/lib/python3.8/multiprocessing/connection.py, line 257, in pollreturn self._poll(timeout)File /opt/conda/lib/python3.8/multiprocessing/connection.py, line 424, in _pollr wait([self], timeout)File /opt/conda/lib/python3.8/multiprocessing/connection.py, line 936, in waittimeout deadline - time.monotonic()File /opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/signal_handling.py, line 66, in handler_error_if_any_worker_fails()
RuntimeError: DataLoader worker (pid 33959) is killed by signal: Killed.
查了一下发现原因是配置文件projects/configs/stage1_track_map/base_track_map.py和projects/configs/stage2_e2e/base_e2e.py里的workers_per_gpu8的设置对我们的服务器来说太多了改为2后再运行上面的命令可以顺利执行完毕。
