序言
fastllm是纯c++实现，无第三方依赖的高性能大模型推理库。
本文以国产海光DCU为例，在AMD ROCm平台下编译部署fastllm以实现LLMs模型推理加速。
测试平台：曙光超算互联网平台SCNet
GPU/DCU：异构加速卡AI 显存64GB PCIE（基于ROCm平台的GPU）
服务器的详细配置，请参考：曙光超算互联网平台SCNet之国产异构加速卡DCU
一、参考资料
fastllm 代码仓库：https://github.com/lcpu-club/fastllm-rocm
二、快速开始
1. fastllm功能概述
- 🚀 纯c++实现，便于跨平台移植，可以在安卓上直接编译。
- 🚀 ARM平台支持NEON指令集加速，X86平台支持AVX指令集加速，NVIDIA平台支持CUDA加速。
- 🚀 支持浮点模型（FP32), 半精度模型(FP16), 量化模型(INT8, INT4) 加速。
- 🚀 支持多卡部署，支持GPU + CPU混合部署。
- 🚀 支持Batch速度优化。
- 🚀 支持并发计算时动态拼Batch。
- 🚀 支持流式输出，很方便实现打字机效果。
- 🚀 支持python调用。
- 🚀 前后端分离设计，便于支持新的计算设备。
- 🚀 目前支持ChatGLM模型，各种LLAMA模型(ALPACA, VICUNA等)，BAICHUAN模型，MOSS模型。
2. 测试环境
root@notebook-1823641624653922306-scnlbe5oi5-42808:/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/build# hipconfig
HIP version : 5.7.24164== hipconfig
HIP_PATH : /opt/dtk/hip
ROCM_PATH : /opt/dtk
HIP_COMPILER : clang
HIP_PLATFORM : amd
HIP_RUNTIME : rocclr
CPP_CONFIG : -D__HIP_PLATFORM_HCC__= -D__HIP_PLATFORM_AMD__= -I"/opt/dtk/hip/include" -I"/opt/dtk/llvm/bin/../lib/clang/15.0.0" -I/opt/dtk/hsa/include== hip-clang
HSA_PATH : /opt/dtk/hsa
HIP_CLANG_PATH : /opt/dtk/llvm/bin
clang version 15.0.0
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/dtk/llvm/bin
AOMP-15.0-61 (http://github.com/ROCm-Developer-Tools/aomp):Source ID:15.0-61-595b0d8133fafef5742f7d39f8e6a07b31afff56LLVM version 15.0.0gitOptimized build.Default target: x86_64-unknown-linux-gnuHost CPU: (unknown)Registered Targets:amdgcn - AMD GCN GPUsr600 - AMD GPUs HD2XXX-HD6XXXx86 - 32-bit X86: Pentium-Pro and abovex86-64 - 64-bit X86: EM64T and AMD64
hip-clang-cxxflags : -std=c++11 -isystem /opt/dtk-24.04/llvm/lib/clang/15.0.0/include/.. -isystem /opt/dtk-24.04/llvm/lib/clang/15.0.0/include/cuda_wrappers -isystem /opt/dtk/hsa/include -isystem /opt/dtk/hip/include -msram-ecc -msram-ecc -msram-ecc -O3
hip-clang-ldflags : --driver-mode=g++ -L"/opt/dtk/hip/lib" -L"/opt/dtk/lib" -O3 -lgcc_s -lgcc -lpthread -lm -lrt=== Environment Variables
PATH=/usr/local/bin:/opt/mpi/bin:/opt/hwloc/bin/:/opt/cmake/bin/:/opt/conda/bin:/opt/conda/bin:/opt/dtk/bin:/opt/dtk/llvm/bin:/opt/dtk/hip/bin:/opt/dtk/hip/bin/hipify:/opt/hyhal/bin:/opt/mpi/bin:/opt/hwloc/bin/:/opt/cmake/bin/:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/conda/bin
LD_LIBRARY_PATH=/usr/local/lib/:/usr/local/lib64/:/opt/mpi/lib:/opt/hwloc/lib:/opt/dtk/hip/lib:/opt/dtk/llvm/lib:/opt/dtk/lib:/opt/dtk/lib64:/opt/hyhal/lib:/opt/hyhal/lib64:/opt/dtk-24.04/hip/lib:/opt/dtk-24.04/llvm/lib:/opt/dtk-24.04/lib:/opt/dtk-24.04/lib64:/opt/hyhal/lib:/opt/hyhal/lib64:/usr/local/lib/:/usr/local/lib64/:/opt/mpi/lib:/opt/hwloc/lib:
HIP_PATH=/opt/dtk/hip== Linux Kernel
Hostname : notebook-1823641624653922306-scnlbe5oi5-42808
Linux notebook-1823641624653922306-scnlbe5oi5-42808 3.10.0-957.el7.x86_64 #1 SMP Fri Oct 25 22:34:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
3. 安装CLBlast
CLBlast: Building and installing
4. cmake编译
# 下载源码
git clone https://github.com/lcpu-club/fastllm-rocm.gitcd fastllm
mkdir build
cd build# 设置环境变量
export ROCM_PATH="/opt/dtk"
export CLBlast_DIR="/usr/lib/x86_64-linux-gnu/cmake/CLBlast"export CXXFLAGS="-I${ROCM_PATH}/include -Wno-pass-failed -D__HIP_PLATFORM_AMD__"# cmake
cmake .. -DUSE_ROCM=ON -DCMAKE_CXX_COMPILER=/opt/dtk/llvm/bin/clang++
root@notebook-1823641624653922306-scnlbe5oi5-42808:/public/home/scnlbe5oi5/Downloads/cache/fastl
lm-rocm/build# cmake .. -DUSE_ROCM=ON -DCMAKE_CXX_COMPILER=/opt/dtk/llvm/bin/clang++
-- The CXX compiler identification is Clang 15.0.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/dtk/llvm/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- USE_CUDA: OFF
-- USE_ROCM: ON
-- PYTHON_API: OFF
-- USE_SENTENCEPIECE: OFF
-- CMAKE_CXX_FLAGS-I/opt/dtk/include -Wno-pass-failed -D__HIP_PLATFORM_AMD__ -pthread --std=c++17 -O3 -march=native
-- The HIP compiler identification is Clang 15.0.0
-- Detecting HIP compiler ABI info
-- Detecting HIP compiler ABI info - done
-- Check for working HIP compiler: /opt/dtk/llvm/bin/clang++ - skipped
-- Detecting HIP compile features
-- Detecting HIP compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Performing Test HIP_CLANG_SUPPORTS_PARALLEL_JOBS
-- Performing Test HIP_CLANG_SUPPORTS_PARALLEL_JOBS - Success
-- System architecture is x86_64
-- Configuring done (5.1s)
-- Generating done (0.6s)
-- Build files have been written to: /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/build
5. make编译
单线程编译
root@notebook-1823641624653922306-scnlbe5oi5-42808:/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/bu
ild# make -j1
[ 2%] Building CXX object CMakeFiles/fastllm.dir/src/fastllm.cpp.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:46: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:32: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:18: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
9 warnings generated.
[ 4%] Building CXX object CMakeFiles/fastllm.dir/src/device.cpp.o
[ 6%] Building CXX object CMakeFiles/fastllm.dir/src/model.cpp.o
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:3:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:100:16: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested herereturn std::unique_ptr<fastllm::basellm> (model);^
1 warning generated.
[ 8%] Building CXX object CMakeFiles/fastllm.dir/src/executor.cpp.o
[ 11%] Building CXX object CMakeFiles/fastllm.dir/src/devices/cpu/cpudevice.cpp.o
[ 13%] Building CXX object CMakeFiles/fastllm.dir/src/devices/cpu/cpudevicebatch.cpp.o
[ 15%] Building CXX object CMakeFiles/fastllm.dir/src/models/chatglm.cpp.o
[ 17%] Building CXX object CMakeFiles/fastllm.dir/src/models/moss.cpp.o
[ 20%] Building CXX object CMakeFiles/fastllm.dir/src/models/llama.cpp.o
[ 22%] Building CXX object CMakeFiles/fastllm.dir/src/models/qwen.cpp.o
[ 24%] Building CXX object CMakeFiles/fastllm.dir/src/models/basellm.cpp.o
[ 26%] Building CXX object CMakeFiles/fastllm.dir/src/models/glm.cpp.o
[ 28%] Building CXX object CMakeFiles/fastllm.dir/src/devices/rocm/rocmdevice.cpp.o
[ 31%] Building CXX object CMakeFiles/fastllm.dir/src/devices/rocm/rocmdevicebatch.cpp.o
[ 33%] Building HIP object CMakeFiles/fastllm.dir/src/devices/rocm/fastllm-rocm.hip.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
4 warnings generated when compiling for gfx906.
4 warnings generated when compiling for gfx926.
4 warnings generated when compiling for gfx928.
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
4 warnings generated when compiling for host.
[ 33%] Built target fastllm
[ 35%] Building CXX object CMakeFiles/main.dir/main.cpp.o
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/main.cpp:1:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/main.cpp:64:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
1 warning generated.
[ 37%] Linking HIP executable main
[ 37%] Built target main
[ 40%] Building CXX object CMakeFiles/quant.dir/tools/src/quant.cpp.o
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/quant.cpp:6:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/quant.cpp:49:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
1 warning generated.
[ 42%] Linking HIP executable quant
[ 42%] Built target quant
[ 44%] Building CXX object CMakeFiles/testOps.dir/test/ops/cppOps.cpp.o
[ 46%] Linking HIP executable testOps
[ 46%] Built target testOps
[ 48%] Building CXX object CMakeFiles/webui.dir/example/webui/webui.cpp.o
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/webui/webui.cpp:4:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/webui/webui.cpp:76:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
1 warning generated.
[ 51%] Linking HIP executable webui
[ 51%] Built target webui
[ 53%] Building CXX object CMakeFiles/benchmark.dir/example/benchmark/benchmark.cpp.o
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/benchmark/benchmark.cpp:5:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/benchmark/benchmark.cpp:82:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
1 warning generated.
[ 55%] Linking HIP executable benchmark
[ 55%] Built target benchmark
[ 57%] Building CXX object CMakeFiles/apiserver.dir/example/apiserver/apiserver.cpp.o
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/apiserver.cpp:4:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/apiserver.cpp:231:8: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested here
struct WorkQueue {^
1 warning generated.
[ 60%] Building CXX object CMakeFiles/apiserver.dir/example/apiserver/json11.cpp.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:251:58: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit JsonString(string &&value) : Value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:259:62: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit JsonArray(Json::array &&value) : Value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:267:64: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit JsonObject(Json::object &&value) : Value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:309:76: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]Json::Json(string &&value) : m_ptr(make_shared<JsonString>(move(value))) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:312:75: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]Json::Json(Json::array &&values) : m_ptr(make_shared<JsonArray>(move(values))) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:314:76: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]Json::Json(Json::object &&values) : m_ptr(make_shared<JsonObject>(move(values))) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:412:29: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]return fail(move(msg), Json());^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value) : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:251:52: note: in instantiation of member function 'json11::Value<json11::Json::STRING, std::basic_string<char>>::Value' requested hereexplicit JsonString(string &&value) : Value(move(value)) {}^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value) : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:259:56: note: in instantiation of member function 'json11::Value<json11::Json::ARRAY, std::vector<json11::Json>>::Value' requested hereexplicit JsonArray(Json::array &&value) : Value(move(value)) {}^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value) : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:267:58: note: in instantiation of member function 'json11::Value<json11::Json::OBJECT, std::map<std::basic_string<char>, json11::Json>>::Value' requested hereexplicit JsonObject(Json::object &&value) : Value(move(value)) {}^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value) : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:272:22: note: in instantiation of member function 'json11::Value<json11::Json::NUL, json11::NullStruct>::Value' requested hereJsonNull() : Value({}) {}^
11 warnings generated.
[ 62%] Linking HIP executable apiserver
[ 62%] Built target apiserver
[ 64%] Building CXX object CMakeFiles/fastllm_tools.dir/src/fastllm.cpp.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:46: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:32: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:18: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
9 warnings generated.
[ 66%] Building CXX object CMakeFiles/fastllm_tools.dir/src/device.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/model.cpp.o
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:3:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:100:16: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested herereturn std::unique_ptr<fastllm::basellm> (model);^
1 warning generated.
[ 71%] Building CXX object CMakeFiles/fastllm_tools.dir/src/executor.cpp.o
[ 73%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/cpu/cpudevice.cpp.o
[ 75%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/cpu/cpudevicebatch.cpp.o
[ 77%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/chatglm.cpp.o
[ 80%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/moss.cpp.o
[ 82%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/llama.cpp.o
[ 84%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/qwen.cpp.o
[ 86%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/basellm.cpp.o
[ 88%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/glm.cpp.o
[ 91%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/rocm/rocmdevice.cpp.o
[ 93%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/rocm/rocmdevicebatch.cpp.o
[ 95%] Building HIP object CMakeFiles/fastllm_tools.dir/src/devices/rocm/fastllm-rocm.hip.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9:/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip ^~~~~~~~~~~
:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
4 warnings generated when compiling for gfx926.
4 warnings generated when compiling for gfx928.
4 warnings generated when compiling for gfx906.
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
4 warnings generated when compiling for host.
[ 97%] Building CXX object CMakeFiles/fastllm_tools.dir/tools/src/pytools.cpp.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/pytools.cpp:78:42: warning: 'make_config' has C-linkage specified, but returns user-defined type 'fastllm::GenerationConfig' which is incompatible with C [-Wreturn-type-c-linkage]DLL_EXPORT fastllm::GenerationConfig make_config(int max_length, bool do_sample, float top_p, int top_k,^
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/pytools.cpp:5:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/pytools.cpp:95:29: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested heremodels.models[id] = fastllm::CreateLLMModelFromFile(path);^
2 warnings generated.
[100%] Linking HIP shared library libfastllm_tools.so
[100%] Built target fastllm_tools
多线程编译
root@notebook-1823641624653922306-scnlbe5oi5-42808:/public/home/scnlbe5oi5/Downloads/cache/fastl
lm-rocm/build# make -j${nproc}
[ 6%] Building CXX object CMakeFiles/fastllm.dir/src/fastllm.cpp.o
[ 17%] Building CXX object CMakeFiles/fastllm.dir/src/device.cpp.o
[ 6%] Building CXX object CMakeFiles/fastllm.dir/src/model.cpp.o
[ 46%] Building CXX object CMakeFiles/fastllm.dir/src/executor.cpp.o
[ 46%] Building CXX object CMakeFiles/fastllm_tools.dir/src/fastllm.cpp.o
[ 48%] Building CXX object CMakeFiles/fastllm.dir/src/devices/cpu/cpudevice.cpp.o
[ 53%] Building CXX object CMakeFiles/fastllm.dir/src/models/chatglm.cpp.o
[ 55%] Building CXX object CMakeFiles/fastllm_tools.dir/src/device.cpp.o
[ 66%] Building CXX object CMakeFiles/fastllm.dir/src/devices/cpu/cpudevicebatch.cpp.o
[ 66%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/rocm/rocmdevicebatch.cpp.o
[ 68%] Building HIP object CMakeFiles/fastllm_tools.dir/src/devices/rocm/fastllm-rocm.hip.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/model.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm.dir/src/models/moss.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/cpu/cpudevice.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm.dir/src/models/basellm.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/executor.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/cpu/cpudevicebatch.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm.dir/src/models/llama.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm.dir/src/models/glm.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm.dir/src/models/qwen.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/chatglm.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/moss.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/qwen.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/llama.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm.dir/src/devices/rocm/rocmdevice.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm.dir/src/devices/rocm/rocmdevicebatch.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/rocm/rocmdevice.cpp.o
[ 68%] Building HIP object CMakeFiles/fastllm.dir/src/devices/rocm/fastllm-rocm.hip.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/glm.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/basellm.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/tools/src/pytools.cpp.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/pytools.cpp:78:42: warning: 'make_config' has C-linkage specified, but returns user-defined type 'fastllm::GenerationConfig' which is incompatible with C [-Wreturn-type-c-linkage]DLL_EXPORT fastllm::GenerationConfig make_config(int max_length, bool do_sample, float top_p, int top_k,^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/pytools.cpp:5:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/pytools.cpp:95:29: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested heremodels.models[id] = fastllm::CreateLLMModelFromFile(path);^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:46: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:32: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:18: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:46: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:32: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:18: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:3:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:100:16: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested herereturn std::unique_ptr<fastllm::basellm> (model);^
2 warnings generated.
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:3:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:100:16: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested herereturn std::unique_ptr<fastllm::basellm> (model);^
1 warning generated.
4 warnings generated when compiling for gfx928.
1 warning generated.
4 warnings generated when compiling for gfx926.
4 warnings generated when compiling for gfx906.
4 warnings generated when compiling for gfx906.
4 warnings generated when compiling for gfx926.
4 warnings generated when compiling for gfx928.
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
4 warnings generated when compiling for host.
4 warnings generated when compiling for host.
9 warnings generated.
9 warnings generated.
[ 71%] Linking HIP shared library libfastllm_tools.so
[ 71%] Built target fastllm_tools
[ 71%] Built target fastllm
[ 77%] Building CXX object CMakeFiles/testOps.dir/test/ops/cppOps.cpp.o
[ 77%] Building CXX object CMakeFiles/benchmark.dir/example/benchmark/benchmark.cpp.o
[ 77%] Building CXX object CMakeFiles/main.dir/main.cpp.o
[ 86%] Building CXX object CMakeFiles/webui.dir/example/webui/webui.cpp.o
[ 86%] Building CXX object CMakeFiles/quant.dir/tools/src/quant.cpp.o
[ 86%] Building CXX object CMakeFiles/apiserver.dir/example/apiserver/apiserver.cpp.o
[ 86%] Building CXX object CMakeFiles/apiserver.dir/example/apiserver/json11.cpp.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:251:58: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit JsonString(string &&value) : Value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:259:62: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit JsonArray(Json::array &&value) : Value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:267:64: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit JsonObject(Json::object &&value) : Value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:309:76: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]Json::Json(string &&value) : m_ptr(make_shared<JsonString>(move(value))) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:312:75: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]Json::Json(Json::array &&values) : m_ptr(make_shared<JsonArray>(move(values))) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:314:76: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]Json::Json(Json::object &&values) : m_ptr(make_shared<JsonObject>(move(values))) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:412:29: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]return fail(move(msg), Json());^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value) : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:251:52: note: in instantiation of member function 'json11::Value<json11::Json::STRING, std::basic_string<char>>::Value' requested hereexplicit JsonString(string &&value) : Value(move(value)) {}^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value) : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:259:56: note: in instantiation of member function 'json11::Value<json11::Json::ARRAY, std::vector<json11::Json>>::Value' requested hereexplicit JsonArray(Json::array &&value) : Value(move(value)) {}^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value) : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:267:58: note: in instantiation of member function 'json11::Value<json11::Json::OBJECT, std::map<std::basic_string<char>, json11::Json>>::Value' requested hereexplicit JsonObject(Json::object &&value) : Value(move(value)) {}^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value) : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:272:22: note: in instantiation of member function 'json11::Value<json11::Json::NUL, json11::NullStruct>::Value' requested hereJsonNull() : Value({}) {}^
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/quant.cpp:6:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/quant.cpp:49:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/main.cpp:1:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/main.cpp:64:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
[ 88%] Linking HIP executable testOps
1 warning generated.
[ 91%] Linking HIP executable quant
[ 91%] Built target testOps
1 warning generated.
[ 93%] Linking HIP executable main
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/benchmark/benchmark.cpp:5:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/benchmark/benchmark.cpp:82:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
[ 93%] Built target quant
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/apiserver.cpp:4:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/apiserver.cpp:231:8: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested here
struct WorkQueue {^
[ 93%] Built target main
1 warning generated.
[ 95%] Linking HIP executable benchmark
[ 95%] Built target benchmark
11 warnings generated.
1 warning generated.
[ 97%] Linking HIP executable apiserver
[ 97%] Built target apiserver
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/webui/webui.cpp:4:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/webui/webui.cpp:76:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
1 warning generated.
[100%] Linking HIP executable webui
[100%] Built target webui
6. 运行demo
准备模型，以 huangyuyang/chatglm2-6b-int4.flm 为例。
# 命令行聊天程序
./main -p chatglm2-6b-int4.flm
root@notebook-1823641624653922306-scnlbe5oi5-42808:/public/home/scnlbe5oi5/Downloads/cache/fastl
ild# ./main -p chatglm2-6b-int4.flm
AVX: ON
AVX2: ON
AARCH64: OFF
Neon FP16: OFF
Neon DOT: OFF
Load (200 / 200)
Warmup...
Launch params (512, 1, 1) are larger than launch bounds (256) for kernel _Z26FastllmRMSNormKerne512EEvPfS0_S0_iif please add __launch_bounds__ to kernel define or use --gpu-max-threads-per-blole program !
finish.
欢迎使用 chatglm 模型. 输入内容对话，reset清空历史记录，stop退出程序.
用户: hi
chatglm: Hello! How can I assist you today?
用户: 中国深圳有哪些旅游景点
chatglm: 深圳是一个美丽的城市,有很多值得一游的景点。以下是一些深圳的旅游景点:1. 深圳湾公园:是一座大型公园,位于深圳湾海滨,可以欣赏到美丽的海景和城市景观。2. 华侨城:这是一个综合性的文化城,里面有很多娱乐设施和景点,如大型主题公园、音乐厅和博物馆等。3. 深圳会展中心:这是一个现代化的展览中心,可以举办各种展览和活动。4. 东部华侨城:这是一个以海洋文化为主题的大型综合性景区,里面有很多海洋生物和各种娱乐项目。5. 深圳中心书城:这是一个大型的书店,里面有很多图书和文化产品。6. 深圳动物园:这是一个大型的动物园,可以了解到各种动物和它们的生活习性。7. 大梅沙海滨公园:这是一个美丽的海滨公园,里面有很多户外活动设施和景观。8. 深圳技术馆:这是一个展示科技和文化的博物馆,可以了解到各种科技发展的趋势和文化。9. 深圳音乐厅:这是一个现代化的音乐厅,可以举办各种音乐演出和活动。10. 深圳海底世界:这是一个大型的海洋生物博物馆,可以了解到各种海洋生物和它们的生活习性。
运行时的资源占用情况
三、FAQ
Q：#error ("Must define exactly one of __HIP_PLATFORM_AMD__ or __HIP_PLATFORM_NVIDIA__"
Must define exactly one of __HIP_PLATFORM_AMD__ or __HIP_PLATFORM_NVIDIA__ Error #1789
/opt/dtk/include/hip/hip_vector_types.h:38:2: error: #error ("Must define exactly one of __HIP_PLATFORM_AMD__ or __HIP_PLATFORM_NVIDIA__");38 | #error("Must define exactly one of __HIP_PLATFORM_AMD__ or __HIP_PLATFORM_NVIDIA__");| ^~~~~
In file included from /opt/dtk/include/hip/hip_runtime.h:113,from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/devices/rocm/fastllm-rocm.hiph:2,from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:38:
/opt/dtk/include/hip/hip_runtime_api.h:8549:61: error: 'hipHostMallocDefault' was not declared in this scope8549 | unsigned int flags = hipHostMallocDefault) {| ^~~~~~~~~~~~~~~~~~~~
/opt/dtk/include/hip/hip_runtime_api.h:8568:61: error: 'hipMemAttachGlobal' was not declared in this scope8568 | unsigned int flags = hipMemAttachGlobal) {| ^~~~~~~~~~~~~~~~~~
make[2]: *** [CMakeFiles/fastllm.dir/build.make:76: CMakeFiles/fastllm.dir/src/fastllm.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:97: CMakeFiles/fastllm.dir/all] Error 2
make: *** [Makefile:91: all] Error 2
错误原因：未指定 D__HIP_PLATFORM_AMD__。
解决方法：
# 设置环境变量
export CXXFLAGS="-I${ROCM_PATH}/include -Wno-pass-failed -D__HIP_PLATFORM_AMD__"
Q：error: use of undeclared identifier 'CUR_DEVICE' contextLayer.ToDevice(CUR_DEVICE)
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/models/chatglm.cpp:382:31: error: use of undeclared identifier 'CUR_DEVICE'hiddenStates.ToDevice(CUR_DEVICE);^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/models/chatglm.cpp:518:34: error: use of undeclared identifier 'CUR_DEVICE'pastKey.ToDevice(CUR_DEVICE);^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/models/chatglm.cpp:519:36: error: use of undeclared identifier 'CUR_DEVICE'pastValue.ToDevice(CUR_DEVICE);^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/models/chatglm.cpp:668:43: error: use of undeclared identifier 'CUR_DEVICE'contextLayer.ToDevice(CUR_DEVICE);^
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:3:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:100:16: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested herereturn std::unique_ptr<fastllm::basellm> (model);^
1 error generated.
make[2]: *** [CMakeFiles/fastllm.dir/build.make:90: CMakeFiles/fastllm.dir/src/device.cpp.o] Error 1
错误原因：未指定 DUSE_ROCM。
解决方法：
cmake .. -DUSE_ROCM=ON -DCMAKE_CXX_COMPILER=/opt/dtk/llvm/bin/clang++
![[机器学习]全景指南：从基础概念到实战流程的全面解析](https://img-blog.csdnimg.cn/img_convert/972f9c11f2b56dfd7cc3d675a8a12433.png)
