ComfyUI-全民舞王-MusePose

musepose-demo

项目介绍

  • https://github.com/TMElyralab/MusePose
  • 发布的模型能够根据给定的姿势序列，生成参考图中人物的舞蹈视频，结果质量超越了同一主题中几乎所有当前开源的模型。
  • 发布pose align算法，以便用户可以将任意舞蹈视频与任意参考图像对齐，这显著提高了推理性能并增强了模型的可用性

部署

cd custom_nodes
git clone https://github.com/TMElyralab/MusePose.git
pip install -r requirements.txt
pip install --no-cache-dir -U openmim 
mim install mmengine 
mim install "mmcv>=2.0.1" 
mim install "mmdet>=3.1.0" 
mim install "mmpose>=1.1.0"

模型下载

cd MusePose

import os
import wget
from tqdm import tqdmos.makedirs('pretrained_weights', exist_ok=True)urls = ['https://download.openmmlab.com/mmdetection/v2.0/yolox/yolox_l_8x8_300e_coco/yolox_l_8x8_300e_coco_20211126_140236-d3bd2b23.pth','https://huggingface.co/yzd-v/DWPose/resolve/main/dw-ll_ucoco_384.pth','https://huggingface.co/TMElyralab/MusePose/resolve/main/MusePose/denoising_unet.pth','https://huggingface.co/TMElyralab/MusePose/resolve/main/MusePose/motion_module.pth','https://huggingface.co/TMElyralab/MusePose/resolve/main/MusePose/pose_guider.pth','https://huggingface.co/TMElyralab/MusePose/resolve/main/MusePose/reference_unet.pth','https://huggingface.co/lambdalabs/sd-image-variations-diffusers/resolve/main/unet/diffusion_pytorch_model.bin','https://huggingface.co/lambdalabs/sd-image-variations-diffusers/resolve/main/image_encoder/pytorch_model.bin','https://huggingface.co/stabilityai/sd-vae-ft-mse/resolve/main/diffusion_pytorch_model.bin']paths = ['dwpose', 'dwpose', 'MusePose', 'MusePose', 'MusePose', 'MusePose', 'sd-image-variations-diffusers/unet', 'image_encoder', 'sd-vae-ft-mse']for path in paths:os.makedirs(f'pretrained_weights/{path}', exist_ok=True)# saving weights 
for url, path in tqdm(zip(urls, paths)):filename = wget.download(url, f'pretrained_weights/{path}')config_urls = ['https://huggingface.co/lambdalabs/sd-image-variations-diffusers/resolve/main/unet/config.json','https://huggingface.co/lambdalabs/sd-image-variations-diffusers/resolve/main/image_encoder/config.json','https://huggingface.co/stabilityai/sd-vae-ft-mse/resolve/main/config.json']config_paths = ['sd-image-variations-diffusers/unet', 'image_encoder', 'sd-vae-ft-mse']# saving config files 
for url, path in tqdm(zip(config_urls, config_paths)):filename = wget.download(url, f'pretrained_weights/{path}')# renaming model name as given in readme
os.rename('pretrained_weights/dwpose/yolox_l_8x8_300e_coco_20211126_140236-d3bd2b23.pth', 'pretrained_weights/dwpose/yolox_l_8x8_300e_coco.pth')

运行效果

在这里插入图片描述

工作流

{"last_node_id": 16,"last_link_id": 11,"nodes": [{"id": 5,"type": "filenamestring","pos": [983,765],"size": {"0": 210,"1": 26},"flags": {},"order": 3,"mode": 0,"inputs": [{"name": "filenames","type": "VHS_FILENAMES","link": 6,"slot_index": 0,"label": "filenames"}],"outputs": [{"name": "STRING","type": "STRING","links": [3],"shape": 3,"slot_index": 0,"label": "STRING"}],"properties": {"Node name for S&R": "filenamestring"}},{"id": 13,"type": "filenamestring","pos": [1241,768],"size": {"0": 210,"1": 26},"flags": {},"order": 6,"mode": 0,"inputs": [{"name": "filenames","type": "VHS_FILENAMES","link": 8,"slot_index": 0,"label": "filenames"}],"outputs": [{"name": "STRING","type": "STRING","links": [9],"shape": 3,"slot_index": 0,"label": "STRING"}],"properties": {"Node name for S&R": "filenamestring"}},{"id": 4,"type": "museposealign","pos": [1077,542],"size": {"0": 315,"1": 150},"flags": {},"order": 4,"mode": 0,"inputs": [{"name": "image","type": "IMAGE","link": 4,"slot_index": 0,"label": "image"},{"name": "video","type": "STRING","link": 3,"widget": {"name": "video"},"slot_index": 1,"label": "video"}],"outputs": [{"name": "IMAGE","type": "IMAGE","links": [7],"shape": 3,"slot_index": 0,"label": "IMAGE"}],"properties": {"Node name for S&R": "museposealign"},"widgets_values": ["",512,700,300,0]},{"id": 10,"type": "VHS_VideoCombine","pos": [371,810],"size": [210,641.7777777777778],"flags": {},"order": 2,"mode": 0,"inputs": [{"name": "images","type": "IMAGE","link": 5,"label": "images"},{"name": "audio","type": "VHS_AUDIO","link": null,"label": "audio"},{"name": "meta_batch","type": "VHS_BatchManager","link": null,"label": "batch_manager"}],"outputs": [{"name": "Filenames","type": "VHS_FILENAMES","links": [6],"shape": 3,"slot_index": 0,"label": "Filenames"}],"properties": {"Node name for S&R": "VHS_VideoCombine"},"widgets_values": {"frame_rate": 24,"loop_count": 0,"filename_prefix": "AnimateDiff","format": "video/h264-mp4","pix_fmt": "yuv420p","crf": 19,"save_metadata": true,"pingpong": false,"save_output": false,"videopreview": {"hidden": false,"paused": false,"params": {"filename": "AnimateDiff_00028.mp4","subfolder": "","type": "temp","format": "video/h264-mp4"}}}},{"id": 11,"type": "VHS_VideoCombine","pos": [973,807],"size": [390,896],"flags": {},"order": 5,"mode": 0,"inputs": [{"name": "images","type": "IMAGE","link": 7,"label": "images"},{"name": "audio","type": "VHS_AUDIO","link": null,"label": "audio"},{"name": "meta_batch","type": "VHS_BatchManager","link": null,"label": "batch_manager"}],"outputs": [{"name": "Filenames","type": "VHS_FILENAMES","links": [8],"shape": 3,"slot_index": 0,"label": "Filenames"}],"properties": {"Node name for S&R": "VHS_VideoCombine"},"widgets_values": {"frame_rate": 24,"loop_count": 0,"filename_prefix": "AnimateDiff","format": "video/h264-mp4","pix_fmt": "yuv420p","crf": 19,"save_metadata": true,"pingpong": false,"save_output": false,"videopreview": {"hidden": false,"paused": false,"params": {"filename": "AnimateDiff_00029.mp4","subfolder": "","type": "temp","format": "video/h264-mp4"}}}},{"id": 6,"type": "LoadImage","pos": [592,1014],"size": {"0": 368.78228759765625,"1": 444.1121826171875},"flags": {},"order": 0,"mode": 0,"outputs": [{"name": "IMAGE","type": "IMAGE","links": [4,10],"shape": 3,"slot_index": 0,"label": "IMAGE"},{"name": "MASK","type": "MASK","links": null,"shape": 3,"label": "MASK"}],"properties": {"Node name for S&R": "LoadImage"},"widgets_values": ["pic (4).png","image"]},{"id": 9,"type": "VHS_LoadVideo","pos": [7,837],"size": [357.83404541015625,876.5938585069445],"flags": {},"order": 1,"mode": 0,"inputs": [{"name": "meta_batch","type": "VHS_BatchManager","link": null,"label": "batch_manager"}],"outputs": [{"name": "IMAGE","type": "IMAGE","links": [5],"shape": 3,"slot_index": 0,"label": "IMAGE"},{"name": "frame_count","type": "INT","links": null,"shape": 3,"label": "frame_count"},{"name": "audio","type": "VHS_AUDIO","links": null,"shape": 3,"label": "audio"},{"name": "video_info","type": "VHS_VIDEOINFO","links": null,"shape": 3,"label": "video_info"}],"properties": {"Node name for S&R": "VHS_LoadVideo"},"widgets_values": {"video": "dance.mp4","force_rate": 40,"force_size": "Disabled","custom_width": 512,"custom_height": 512,"frame_load_cap": 0,"skip_first_frames": 100,"select_every_nth": 1,"choose video to upload": "image","videopreview": {"hidden": false,"paused": false,"params": {"frame_load_cap": 0,"skip_first_frames": 100,"force_rate": 40,"filename": "dance.mp4","type": "input","format": "video/mp4","select_every_nth": 1}}}},{"id": 14,"type": "VHS_VideoCombine","pos": [1376,803],"size": [390,896],"flags": {},"order": 8,"mode": 0,"inputs": [{"name": "images","type": "IMAGE","link": 11,"label": "images"},{"name": "audio","type": "VHS_AUDIO","link": null,"label": "audio"},{"name": "meta_batch","type": "VHS_BatchManager","link": null,"label": "batch_manager"}],"outputs": [{"name": "Filenames","type": "VHS_FILENAMES","links": [],"shape": 3,"slot_index": 0,"label": "Filenames"}],"properties": {"Node name for S&R": "VHS_VideoCombine"},"widgets_values": {"frame_rate": 60,"loop_count": 0,"filename_prefix": "AnimateDiff","format": "video/h264-mp4","pix_fmt": "yuv420p","crf": 19,"save_metadata": true,"pingpong": false,"save_output": false,"videopreview": {"hidden": false,"paused": false,"params": {"filename": "AnimateDiff_00034.mp4","subfolder": "","type": "temp","format": "video/h264-mp4"}}}},{"id": 12,"type": "musepose","pos": [1465,509],"size": {"0": 315,"1": 246},"flags": {},"order": 7,"mode": 0,"inputs": [{"name": "image","type": "IMAGE","link": 10,"label": "image"},{"name": "video","type": "STRING","link": 9,"widget": {"name": "video"},"label": "video"}],"outputs": [{"name": "IMAGE","type": "IMAGE","links": [11],"shape": 3,"slot_index": 0,"label": "IMAGE"}],"properties": {"Node name for S&R": "musepose"},"widgets_values": ["",512,512,1000,48,4,3,10,12]}],"links": [[3,5,0,4,1,"STRING"],[4,6,0,4,0,"IMAGE"],[5,9,0,10,0,"IMAGE"],[6,10,0,5,0,"VHS_FILENAMES"],[7,4,0,11,0,"IMAGE"],[8,11,0,13,0,"VHS_FILENAMES"],[9,13,0,12,1,"STRING"],[10,6,0,12,0,"IMAGE"],[11,12,0,14,0,"IMAGE"]],"groups": [],"config": {},"extra": {"ds": {"scale": 0.6303940863128529,"offset": {"0": 746.0277438571095,"1": 34.665619475590915}}},"version": 0.4
}

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：http://www.rhkb.cn/news/349456.html

如若内容造成侵权/违法违规/事实不符，请联系长河编程网进行投诉反馈email:809451989@qq.com，一经查实，立即删除！

相关文章

第二届人工智能、系统与网络安全国际学术会议 (AISNS 2024）

第二届人工智能、系统与网络安全国际学术会议 (AISNS 2024）

第二届人工智能、系统与网络安全国际学术会议 (AISNS 2024） 2024 2nd International Conference on Artificial Intelligence, Systems and Network Security 一、重要信息 大会官网：www.aisns.org （点击参会/投稿/了解会议详情&#xff09…
阅读更多...
基于GTX 8B10B编码的自定义PHY上板测试（高速收发器十四）

基于GTX 8B10B编码的自定义PHY上板测试（高速收发器十四）

前文整理了GTX IP，完成了自定义PHY协议的收发模块设计，本文将通过光纤回环，对这些模块上板测试，首先需要编写一个用于生成测试数据的用户模块。 1、测试数据生成模块 本模块用于生成自定义PHY协议的测试数据，通过axi_…
阅读更多...
什么是网站页面组件，有什么用处

什么是网站页面组件，有什么用处

1.组件是页面的组成部分 网站的每个页面都是由每个小组件拼接而成的，当然了，有一些网站并不是用小组件拼接的，但是像这些网站它是固定住标签，然后用js直接调用数据显示，这个就是定制化的体现。像那些用小组件组…
阅读更多...
Elastic 索引结构-倒排索引

Elastic 索引结构-倒排索引

前言 Elastic 在数据库分类中一般被分为全文检索的数据库，那为什么这么区分呢？主要是因为其独特的索引结构 即倒排索引。 倒排索引 倒排索引先将文档中包含的关键字全部提取出来，然后再将关键字与文档的对应关系保存起来，最后再…
阅读更多...
多模态大模型：识别和处理图片与视频的技术详解

多模态大模型：识别和处理图片与视频的技术详解

多模态大模型：识别和处理图片与视频的技术详解 多模态大模型：识别和处理图片与视频的技术详解1. 什么是多模态大模型？2. 多模态大模型的基本架构3. 识别和处理图片3.1 图像特征提取3.2 图像分类与识别3.3 图像生成与增强 4. 识别和处理视频4.…
阅读更多...
WebSocket 快速入门 与 应用

WebSocket 快速入门 与 应用

WebSocket 是一种在 Web 应用程序中实现实时、双向通信的技术。它允许客户端和服务器之间建立持久性的连接，以便可以在两者之间双向传输数据。 以下是 WebSocket 的一些关键特点和工作原理： 0.特点： 双向通信：WebSocket 允许服务…
阅读更多...
什么是端口转发？路由器如何正确的设置端口转发和范围转发？（外网访问必备设置）

什么是端口转发？路由器如何正确的设置端口转发和范围转发？（外网访问必备设置）

文章目录 📖 介绍 📖🏡 演示环境 🏡📒 端口转发 📒🚀 端口转发的应用场景💡 路由器如何设置端口转发（示例）💡 端口范围转发（示例）🎯 范围转发的应用场景🛠️ 设置范围转发📝 范围转发实操示例🎈 注意事项 🎈⚓️ 相关链接 ⚓️📖 介绍 📖 …
阅读更多...
遗传算法求解车间调度问题(附python代码)

遗传算法求解车间调度问题(附python代码)

背景介绍 车间调度问题（Job Shop Scheduling Problem, JSSP）是一类经典的组合优化问题，它在制造业和生产管理中有着广泛的应用。JSSP 的目标是对车间中的一系列作业进行排程，以使得作业在不同机器上的加工顺序是最优的&#xff0…
阅读更多...
什么是场外期权？场外期权有几种做法？

什么是场外期权？场外期权有几种做法？

今天带你了解什么是场外期权？场外期权有几种做法？期权分为场内期权，场外期权。场内期权我们都知道，是在期货盘里购买的期权，但场外期权呢？ 什么是场外期权？ 场外期权是一种在交易所之外进行交易…
阅读更多...
WinForm之TCP服务端

WinForm之TCP服务端

目录 一 原型 二 源码 一 原型 二 源码 using System.Net; using System.Net.Sockets; using System.Text;namespace TCP网络服务端通讯 {public partial class Form1 : Form{public Form1(){InitializeComponent();}TcpListener listener null;TcpClient handler null;Ne…
阅读更多...
【Linux】从零开始配置新的服务器的机器学习环境

【Linux】从零开始配置新的服务器的机器学习环境

终端远程登录 ssh -p [端口号] [服务器用户名][服务器IP]或者 ssh [用户名][主机地址]第二种的前提是在.ssh\config中配置了host 安装文本编辑器vim 主要用于后续的文本编辑，个人比较习惯用vim，根据自己喜好选择 更新apt sudo apt update安装文本编辑…
阅读更多...
陪诊小程序开发，陪诊师在线接单

陪诊小程序开发，陪诊师在线接单

近几年，陪诊师成为了一个新兴行业，在科技时代中，陪诊小程序作为互联网下的产物，为陪诊市场带来了更多的便利。 当下生活压力大，老龄化逐渐严重，年轻人很难做到陪同家属看病。此外，就诊中出现了…
阅读更多...
【npm】安装报错（大多是版本冲突）

【npm】安装报错（大多是版本冲突）

1.eslint版本太高，Vuex报错 我使用的是vue2.x，npm 安装Vuex3版本，控制台报错说eslint版本太高 解决办法：Vue2项目安装 vuex3 出现版本报错解决方法_安装 vuex3版本 eslint 报错-CSDN博客
阅读更多...
在Lua解释器中注册自定义函数库

在Lua解释器中注册自定义函数库

本文目录 1、引言2、注册原理3、实例4、程序验证 文章对应视频教程： 暂无，可以关注我的B站账号等待更新。 点击图片或链接访问我的B站主页~~~ 1、引言 在现代软件开发中，Lua因其轻量级、高效和可嵌入性而被广泛使用。作为一种灵活的脚本语言…
阅读更多...
MyBatis使用 PageHelper 分页查询插件的详细配置

MyBatis使用 PageHelper 分页查询插件的详细配置

1. MyBatis使用 PageHelper 分页查询插件的详细配置 文章目录 1. MyBatis使用 PageHelper 分页查询插件的详细配置2. 准备工作3. 使用传统的 limit 关键字进行分页4. PageHelper 插件（配置步骤）4.1 第一步：引入依赖4.2 第二步：在m…
阅读更多...
【SpringBoot整合系列】SpringBoot整合kinfe4j

【SpringBoot整合系列】SpringBoot整合kinfe4j

目录 kinfe4j与Swagger的区别 SpringBoot2.x整合kinfe4j1.添加依赖2.启动类注解3.创建Knife4J配置类4.实体类5.接口admin访问 api访问 常用注解汇总SpringBoot3.x整合Kinfe4j启动报错解决1.更换依赖2.启动类3.配置4.配置类5.参数实体类6.接口admin访问 api访问 各版本注解参照 …
阅读更多...
视频美颜工具技术探秘：直播美颜SDK的应用与发展

视频美颜工具技术探秘：直播美颜SDK的应用与发展

今天，笔者将深入探讨直播美颜SDK的应用场景和发展趋势，揭示其背后的技术奥秘和潜力。 一、直播美颜SDK的基本原理 直播美颜SDK其基本原理包括以下几个方面： 人脸检测与特征定位 肤色分析与调整 瑕疵修复与细节增强 滤镜和特效应用 二、…
阅读更多...
【算法】Graham 凸包扫描算法 ( 凸包概念 | 常用的凸包算法 | 角排序 | 叉积 | Python 代码示例 )

【算法】Graham 凸包扫描算法 ( 凸包概念 | 常用的凸包算法 | 角排序 | 叉积 | Python 代码示例 )

文章目录 一、Graham 凸包扫描算法1、凸包概念2、常用的凸包算法3、Graham 凸包扫描算法 二、Graham 算法前置知识点1、角排序2、叉积3、算法过程分析 三、代码示例1、完整代码示例2、执行结果 使用 Graham 算法绘制的凸包效果 : 博客代码下载 : https://download.csdn.net/d…
阅读更多...
算法刷题【二分法】

算法刷题【二分法】

题目： 注意题目中说明了数据时非递减的，那么这样就存在二分性，能够实现logn的复杂度。二分法每次只能取寻找特定的某一个值，所以我们要分别求左端点和有端点。 分析第一组用例得到结果如下: 成功找到左端点8 由此可知&#xff0…
阅读更多...
Spring Boot集成 Spring Retry 实现容错重试机制并附源码

Spring Boot集成 Spring Retry 实现容错重试机制并附源码

😄 19年之后由于某些原因断更了三年，23年重新扬帆起航，推出更多优质博文，希望大家多多支持～ 🌷 古之立大事者，不惟有超世之才，亦必有坚忍不拔之志 🎐 个人CSND主页——Mi…
阅读更多...
最新文章
推荐文章

Copyright @ 2022~2023