ComfyUI X UnrealEngine

오늘의 주제

Unreal Engine 의 프로젝트에 ComfyUI를 플러그인으로 연결해주기

git  : ComfyTextures

https://github.com/AlexanderDzhoganov/ComfyTextures

GitHub - AlexanderDzhoganov/ComfyTextures: Unreal Engine ⚔️ ComfyUI - Automatic texturing using generative diffusion models

 

ComfyUI란 ?
- Stable Diffusion 기반 생성형 AI 로서, 오픈소스,노드기반 구조라서 확장이 매우 용이하다.

 

미리 보는 결과물

버튼 한번으로 언리얼 텍스쳐,머티리얼 수정

 

원리 ? 

씬 안에있는 카메라의 시점에서 보이는 물체들에 대해

프롬프트 기반으로 생성된 AI 이미지를 투사해 오브젝트에 텍스쳐나 머티리얼을 입히는 과정

 

실행 환경

CPU : I9-9900K , GPU : RTX 2080ti

운영체제 Windows 10 64bit

프로그램 버전

Stability Matrix 2.10.1,    Python ver 3.10.11

ComfyUI Revision: 2121 [831511a1] | Released on '2024-04-09'

 

UnrealEngine 5.3.2

VisualStudio 2022  _ 17.9.6

 

사용순서

1. ComfyUI와 언리얼엔진을 설치한다.

2. 해당 깃에서 git clone 을 이용하여 프로젝트를 복제한다.

(만약 git clone을 하지않고 윈도우 압축파일로 받아서 압축해제로 푼다면 git lfs 파일이 제대로 넘어오지 않을 수 있다.)

3. 프로젝트(uproject)에 우클릭하여 generate 버튼으로 프로젝트를 빌드한다.
(VisualStudio 버전을 언리얼 버전과 잘 맞춰주어야 가능하다. - 이슈 1)

4. 컨텐츠브라우저에서 플러그인 에셋이 보이는지 확인, 유틸리티 위젯을 띄워준다.

세부설명은 더보기 참고

 

컨텐츠브라우저에서 플러그인 에셋 확인방법

컨텐츠브라우저 - 플러그인에셋 보여주기 활성화

유틸리티위젯 활성화

툴 - 유틸리티위젯 활성화

Tools - Editor Utilty Widgets - Comfy Textures Widget

5. ComfyUI의 Models 폴더 또는 Stability의 Models 폴더(Stability Matrix 프로그램 사용시) 에 필요한 모델들을 넣어준다.

세부설명은 더보기 참고

 

Git - Model Download Link

모델 다운로드 링크이며,

sdxl 체크포인트 두개와, sdxl기반 컨트롤넷 canny,depth 그리고 lora , upscale

6개의 모델이 있다.

 

본인의 경우
ex) ComfyUI 폴더에 넣을 시
D:\StabilityMatrix-win-x64\Data\Packages\ComfyUI\models

경로에 들어가서 폴더명에 맞게 넣어준다.

ex) Stability 폴더에 넣을 시
D:\StabilityMatrix-win-x64\Data\Models

경로에 들어가서 폴더명에 맞게 넣어준다.

 

Stability 의 모델 폴더에 넣을 시 해당 폴더를 매트릭스 프로그램 모두가 공유 하기 때문에
ComfyUI 말고 다른 패키지(ex - StableDiffusion Web UI, ...) 들에서도 사용할 수 있다. 

6. ComfyUI를 실행한 채, UE에서는 Room을 클릭한채 WBP의 Render 버튼을 누른다.

만약 Render 버튼이 보이지않는다면

ComfyUI 가 켜져있지않거나, 연결이 제대로 되어있지않다는것.

 

이슈 1. VS를 최신버전 (17.9.6) 으로 사용할때 언리얼엔진에서 빌드가 안되는현상 (스크립트 두개 수정)

https://forums.unrealengine.com/t/error-compiling-the-automation-tool-after-updating-visual-studio-today-unreal-5-3-2/1393088/5

Error compiling the Automation Tool after updating Visual Studio today (Unreal 5.3.2)

 

 

 

 

 

플러그인 패키징 관련

(추가할 예정)

이슈 : CodeError-1 또는 CodeError-6 이 뜬다면 참고할 페이지

세부설명은 더보기 참고

UATHelper: Package Plugin Task (Windows): Running AutomationTool...
UATHelper: Package Plugin Task (Windows): Using bundled DotNet SDK version: 6.0.302
UATHelper: Package Plugin Task (Windows): Starting AutomationTool...
UATHelper: Package Plugin Task (Windows): Parsing command line: BuildPlugin -Plugin=D:/UE/MyProjects/Plugins/ComfyTextures/ComfyTextures.uplugin -Package=D:/UE/MyProjects/Plugins/ComfyTextures/ComfyTextures -CreateSubFolder -nocompile -nocompileuat
UATHelper: Package Plugin Task (Windows): Initializing script modules...
UATHelper: Package Plugin Task (Windows): Total script module initialization time: 0.29 s.
UATHelper: Package Plugin Task (Windows): Executing commands...
UATHelper: Package Plugin Task (Windows): Copying 45 file(s) using max 64 thread(s)
UATHelper: Package Plugin Task (Windows): Reading plugin from D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\ComfyTextures.uplugin...
UATHelper: Package Plugin Task (Windows): Building plugin for host platforms: Win64
UATHelper: Package Plugin Task (Windows): Running: C:\Program Files\Epic Games\UE_5.3\Engine\Binaries\ThirdParty\DotNet\6.0.302\windows\dotnet.exe "C:\Program Files\Epic Games\UE_5.3\Engine\Binaries\DotNET\UnrealBuildTool\UnrealBuildTool.dll" UnrealEditor Win64 Development -Project="D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\HostProject.uproject" -plugin="D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\ComfyTextures.uplugin" -noubtmakefiles -manifest="D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Saved\Manifest-Unr
ealEditor-Win64-Development.xml" -nohotreload -log="C:\Users\it\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_5.3\UBT-UnrealEditor-Win64-Development.txt"
UATHelper: Package Plugin Task (Windows): Log file: C:\Users\it\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_5.3\UBT-UnrealEditor-Win64-Development.txt
UATHelper: Package Plugin Task (Windows): Parsing headers for UnrealEditor
UATHelper: Package Plugin Task (Windows):   Running Internal UnrealHeaderTool D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\HostProject.uproject D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Intermediate\Build\Win64\UnrealEditor\Development\UnrealEditor.uhtmanifest -WarningsAsErrors -installed
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(97): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(100): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(103): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(112): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(115): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(118): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(121): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(124): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(151): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(154): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(157): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(160): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(163): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(166): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(169): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(172): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(175): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(178): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(187): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(190): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(193): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(196): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(199): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(202): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): Total of 0 written
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(221): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(224): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(227): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(230): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(233): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(236): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(245): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(248): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): D:\UE\MyProjects\Plugins\ComfyTextures\ComfyTextures\HostProject\Plugins\ComfyTextures\Source\ComfyTextures\Public\ComfyTexturesWidgetBase.h(251): Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module.
UATHelper: Package Plugin Task (Windows): Total execution time: 5.76 seconds
UATHelper: Package Plugin Task (Windows): Took 5.88s to run dotnet.exe, ExitCode=6
UATHelper: Package Plugin Task (Windows): UnrealBuildTool failed. See log for more details. (C:\Users\it\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_5.3\UBT-UnrealEditor-Win64-Development.txt)
UATHelper: Package Plugin Task (Windows): AutomationTool executed for 0h 0m 6s
UATHelper: Package Plugin Task (Windows): AutomationTool exiting with ExitCode=6 (6)
UATHelper: Package Plugin Task (Windows): BUILD FAILED

https://upbo.tistory.com/122

[Unreal Engine] 플러그인 패키징 시 블루프린트 호출 가능한 함수에 카테고리 지정자가 필요하다는