专栏算法工具链【求助】征程6 在使用imagenet数据打包lmdb时报错!

【求助】征程6 在使用imagenet数据打包lmdb时报错!

已解决
珠穆朗玛峰修电梯2024-12-17
59
4

路径:/user/horizon_j6_open_explorer/samples/ai_toolchain/horizon_model_train_sample/scripts> 使用命令:python3 tools/datasets/imagenet_packer.py --src-data-dir data/ --target-data-dir data/ --split-name train --num-workers 10 --pack-type lmdb

 

报错:

/usr/local/lib/python3.10/dist-packages/horizon_plugin_pytorch/torch_patch.py:13: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. _register_pytree_node(slice, _slice_flatten, _slice_unflatten) `aidisdk` dependency is not available. Loading dataset from data/ Traceback (most recent call last): File "/horizon_j6_open_explorer/samples/ai_toolchain/horizon_model_train_sample/scripts/tools/datasets/imagenet_packer.py", line 54, in packer = ImageNetPacker( File "/usr/local/lib/python3.10/dist-packages/hat/utils/package_helper.py", line 243, in wrapper return func(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/hat/data/datasets/imagenet.py", line 123, in __init__ self.dataset = torchvision.datasets.ImageNet( File "/usr/local/lib/python3.10/dist-packages/torchvision/datasets/imagenet.py", line 53, in __init__ self.parse_archives() File "/usr/local/lib/python3.10/dist-packages/torchvision/datasets/imagenet.py", line 66, in parse_archives parse_devkit_archive(self.root) File "/usr/local/lib/python3.10/dist-packages/torchvision/datasets/imagenet.py", line 147, in parse_devkit_archive _verify_archive(root, file, md5) File "/usr/local/lib/python3.10/dist-packages/torchvision/datasets/imagenet.py", line 103, in _verify_archive raise RuntimeError(msg.format(file, root)) RuntimeError: The archive ILSVRC2012_devkit_t12.tar.gz is not present in the root directory or is corrupted. You need to download it externally and place it in data/.

数据放在/ user/horizon_j6_open_explorer/samples/ai_toolchain/horizon_model_train_sample/scripts/data

data目录中有ILSVRC2012_devkit_t12.tar.gz,ILSVRC2012_img_train.tar,ILSVRC2012_img_val.tar

有大哥知道为啥嘛

算法工具链
征程6技术深度解析
评论2
0/1000
  • Huanghui
    Lv.5

    收到,先看看

    2024-12-17
    0
    0
  • Huanghui
    Lv.5
    RuntimeError: The archive ILSVRC2012_devkit_t12.tar.gz is not present in the root directory or is corrupted. You need to download it externally and place it in data/.

    猜测一下:会不会是你下载的 ILSVRC2012_devkit_t12.tar.gz 这个文件不完整呢,你试着解压一下看看是不是可以解压,或看一下md5是不是匹配
    2024-12-17
    0
    2
    • 珠穆朗玛峰修电梯回复Huanghui:

      是的,我是使用tinyimagenet数据,转换了imagenet里的数据格式不匹配,导致这个错误,应该还是需要下载原始的imagenet数据集

      2024-12-17
      0
    • Huanghui回复珠穆朗玛峰修电梯:

      那就下载呗,祝你好运,ImageNet数据集太大了!

      2024-12-17
      0