StreamDiffusionTD 0.2.3🚀🖼️ [ Changelog ]
Added 2024-10-09 01:18:01 +0000 UTCGet the TOX here >>> https://www.patreon.com/posts/113644784/
Fixes:
Resolved common installation issue - FIXED "module 'huggingface_hub.utils' has no attribute '_errors'" - this should no longer be an issue. No need to reinstall if you have been stuck on this.
Fixed SD-Turbo specific LoRA weight adjustment for real-time changes
Fixed v2v Loading and optimized performance in main stream loop
Improvements:
ControlNet support for SDXL models added via "Download ControlNet" option (openpose, canny, scribble, depth)
reminder - other ControlNets can be loaded via local files + Huggingface IDs via the Select Controlnet file option
Main SD Model Dropdown will see files in the model folder
Added Sdmodelsfolder parameter (default to models/Model)
You can set this parameter any folder on your computer to the model files inside.
New model folder - models/checkpoints - will also be looked at for main SD models. I suggest downloading to this folder
FPS counter added when operator is running in the UI icons
Small qol improvements throughout
Comments
hey i am having the same problem.... Were u able to resolve it and if yes how ?
vaishnavi shevde
2025-06-29 07:06:03 +0000 UTCI have a problem is : TensorRT is not found in the virtual environment
AZIZA ABDELHAK
2025-06-11 14:08:49 +0000 UTCEagerly waiting for this week's update!
Daniel Gary
2025-02-12 21:44:50 +0000 UTCHeyHo, as I start a new realtime AI project these days: any news on the new version. I am really curious about what you have been cooking and if it's worth to wait some days before I start.
Christopher Lensing
2025-02-12 17:32:54 +0000 UTCAlso, can I get a discord invite?
Hardik Dhingra
2025-02-11 10:49:25 +0000 UTCHey! been trying the streamdiff tool. It's pretty amazing. got an issue though. tensorrt doesn't work after changing the resolution. default 512*512 resolution works fine though. is there any workaround for this?
Hardik Dhingra
2025-02-11 10:49:01 +0000 UTCdid u ever get this resolved?
mr Zpacely
2025-02-11 01:46:11 +0000 UTCHey guys, is there a video about huggingface / other models / lora / checkpoints downloaded from different sites (civitai, ecc) ?? I'm really curious about experimenting, and the base SD model from the component works but I'd love to customize more with other models
Stefano Settembrini
2025-02-10 11:02:34 +0000 UTCIs it possible to choose which GPU it uses if I have more than one
seboya
2025-02-07 09:31:43 +0000 UTCWonderful! Looking forward to it!!!
Daniel Gary
2025-02-07 09:26:57 +0000 UTCPERFECT!!!
Daniel Gary
2025-02-07 09:26:28 +0000 UTChey ! ive got a new version coming out this week. And tensorrt supports up to 1024 per side.
dotsimulate
2025-02-07 04:10:38 +0000 UTCthis week. putting final touches / release notes together currently.
dotsimulate
2025-02-07 04:08:45 +0000 UTCsupporting python 3.11.9 on the next version (and still compatible with 3.10.9) cuda support stays on 11.8/12.1 at the moment.
dotsimulate
2025-02-07 04:07:57 +0000 UTCYes ive got an update coming this week. Just working on documentation and final fixes for apple support.
dotsimulate
2025-02-07 04:06:12 +0000 UTCWas wondering if there is going to be an update for compability with new cuda and python versions?
Sergej G
2025-02-06 20:32:07 +0000 UTCIs this project abandoned? If not, when is the next release?
Daniel Gary
2025-02-01 12:35:41 +0000 UTC"Add Python, CUDA, and Git to your system's PATH during installation." What does this mean? I just installed it normally to my computer system. What else do i need to do? I didn't see any questions like that when i was installing all three components... Please help me
Leon Krykhtin
2025-01-30 06:14:42 +0000 UTCHello :) Where can I install new models?
Dharuma Trejo
2025-01-29 17:34:20 +0000 UTCLoving this component so far! Great work. I'm interested in training a custom LoRA - any advice on how to get started? Also, can you please share any other LoRA known to work well with this TOX?
Oscar Ingham
2025-01-29 05:11:19 +0000 UTCI would check your huggingface folder location on the installation page.
Nick Bogannam
2025-01-27 04:59:06 +0000 UTCHello, I am trying to change the 'SD model id/ path' on the stream setting page to that of a model stored locally on my computer, however it is unable to run giving me an error that it needs to be a URL path to a huggingface model. Any direction of this would greatly be apreciated
Miranda
2025-01-25 12:14:52 +0000 UTC:(
Daniel Gary
2025-01-20 01:26:35 +0000 UTCWhen is the new version coming out? What's the newest and highest resolution model that I can run with the current (and upcoming) release, with tensorrt enabled?
Daniel Gary
2025-01-17 09:13:46 +0000 UTCit works now but i loocking for a solution the model is taken so many GB when i hit pulse "startstream"
Pascal P
2025-01-16 22:10:37 +0000 UTCWhat's the newest and highest resolution model that I can run with the current (and upcoming) release, with tensorrt enabled?
Daniel Gary
2025-01-16 01:03:41 +0000 UTCsdxl turbo did not work for me with tensorrt, normally it works.. Also, almost none of the models that I get from civit.ai work - they either throw and error or generate bad output
Daniel Gary
2025-01-16 01:02:59 +0000 UTCwhich new models? some recent bigger image models have a different architecture and are quite heavy - like sd 3.5 and flux which won't be supported in next release. but things are changing always and im keeping my eye out.
dotsimulate
2025-01-16 00:33:38 +0000 UTCNew version? We can't use new models with this one. Will the new version support new models?
Daniel Gary
2025-01-16 00:07:09 +0000 UTCBefore installing tensorRT try to Pulse Manual Activate VENV and execute the following commang in terminal: pip install --upgrade huggingface_hub==0.24 after that tensorRT will install ok.
Roman Rusavsky
2025-01-13 07:29:09 +0000 UTCHi Mate! Thank you for version 0.2.3. However, I'm encountering an issue with TensorRT—it seems to be missing in the virtual environment. I need to resolve this. Do you have any tips or tricks to fix it? Please let me know. Thanks! have a nice day!
Andrea
2025-01-10 15:39:50 +0000 UTCThank you, artist. To use the 0.2.3 version, do I not proceed to the Cuda 12.6 version, but to use the latest version, do I need to change it to 12.1 and install it? Or will it proceed with 12.6 version without any problems?
구도의밭
2025-01-09 09:27:39 +0000 UTCHello, this is a wonderfull TOX! Quick question: Is it possible to use the new SD 3.5 large turbo model?
Lukas Matthys
2025-01-08 13:10:54 +0000 UTCI was stuck here too. The .tox install pulses seem to use the latest python version installed. What finally worked for me was uninstalling everything leaving only 3.10, then the 2. Install [ venv + all req ] step completes without errors for torch and then the 3. Install TensorRT will complete. I also opened the venv in Powershell and ran "pip install tensorrt-cu11 tensorrt-lean-cu11", to see if I needed to install TensorRT according to this page: https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html but I'm not sure if that is related
Jeroen Hofs
2025-01-06 13:57:31 +0000 UTCGreat news. I played around with SD TD after quite some time the other day and was wondering if there is still something going on with this tool, as it is so very, very cool to play with. No intention to push you included. Stream Diffusion itself seems to be out of development for month now and I was wondering if everyone just left the party for some other stuff....
Christopher Lensing
2024-12-28 19:57:01 +0000 UTCYep! Ive been working on revamped mac support and embeddings for streamdiffusion (coming in about a week once I can record a video). Mostly just been busy due to recently moving apartments and traveling most of November/December. also have been developing a full new operator class that has been taking more dev time.
dotsimulate
2024-12-28 17:34:10 +0000 UTCI appreciate this and your work so much. Thanks for giving us this, Dot.
monokuromasu
2024-12-27 18:34:54 +0000 UTCIs this actually still a thing. Do you still work on this?
Christopher Lensing
2024-12-27 17:54:51 +0000 UTChey ! this link should work - https://discord.gg/mk2Tc9wM
dotsimulate
2024-12-26 19:57:06 +0000 UTCDoes anyone know how I can join the Discord Group? I paid for patreon so that I could use Stream Diffusion but it wont even install because of something called TensorRT. I googled it but now I have to know how to use coding to install from github. I dont understand any of it.
Keli'i Centurion
2024-12-20 02:11:26 +0000 UTCWhen I pulse Start Stream I got the following error: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /CompVis/stable-diffusion/main/configs/stable-diffusion/v1-inference.yaml (Caused by NewConnectionError(': Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))
Andrei Ionut Apopei
2024-12-19 09:06:46 +0000 UTCHo lo stesso identico problema
Mattia Piatti
2024-12-18 13:14:23 +0000 UTCI'm trying to use this version but the only way can get it to work is using 144x144 pixels, and tensorrt is kinda weird, when it works it says that it does not have enough memory, when it doesn't the images start to generate (just in 144x144px)
joão penna
2024-12-14 02:42:43 +0000 UTCHi everyone, I'm trying to download streamdiffusion I'm just missing the tesorRT, the problem is that when I try to download it it gives me this error: from huggingface_hub import HfFolder, cached_download, hf_hub_download, model_info ImportError: cannot import name 'cached_download' from 'huggingface_hub' (C:\StreamD\StreamDiffusion\venv\lib\site-packages\huggingface_hub\__init__.py). Has this happened to anyone? Do you know how to fix it? Because without downloading this it doesn't even let me start the prompts. If I try to start one it still tries to download the tesorRT without success.
Alessandro Sasso
2024-12-11 08:54:43 +0000 UTChow do you fix this issue ?
Justin
2024-11-30 23:49:44 +0000 UTChey guys! i was using the operator pretty smoothly, but it seems to be stuck when building the tensorrt engine. I know it takes a while but ive waited now way longer than i used to. tried building an engine for 512 x 512 and then 960 x 512 but no luck in either.
Adrian Stein
2024-11-30 16:11:18 +0000 UTCsame issue
Christopher Davenport
2024-11-28 14:28:32 +0000 UTCSeems like Hugging Face might change API links. There is problem then installing TensorRT on a new location, (ImportError: cannot import name 'cached_download' from 'huggingface_hub' , Did you mean: 'hf_hub_download') also there is new error, happend when streamdiffusion install ( subprocess-exited-with-error × git clone --filter=blob:none --quiet https://github.com/dotsimulate/StreamDiffusion.git ) - when you install from a this tox into new location also.
Tony Q
2024-11-23 23:09:10 +0000 UTCThe same issue with 0.2.3
Serhii Panin
2024-11-23 21:05:05 +0000 UTCHellooo :) How can i add a negative prompt? :)
Daniel Heitmüller
2024-11-23 14:39:33 +0000 UTCHi there! Sorry if this is a silly question, but how can I output an image with the same dimensions as my input? My output size is currently fixed at 512x512 px, but I’d like to output an image at 1920x1080.
Yiwen Wang
2024-11-14 14:35:42 +0000 UTCHey there - how is this going? Would love to get this working on Mac. Hope all is well!
A Olis
2024-11-13 16:21:55 +0000 UTCHas anyone figured out how to enable the safety checker? Trying to keep it very strict SFW
Josh Mutscheller
2024-11-12 23:12:14 +0000 UTCI'm failing at getting StreamDiffusionTD to behave with either SDXL 1.5 or SDXL-Turbo, with xformers. I managed to patch StreamDiffusion myself for a Linux build of it, and it works over there, but it looks like the LCM-LoRA doesn't get loaded (I also tried giving a custom one): I need a very large step count for the model to produce an image corresponding to the input image, and it doesn't appear to follow the prompt. Swapping back to SD-Turbo makes it all work again. Are there instructions somewhere for how to configure the node to work with SDXL 1.5 or SDXL-Turbo?
Matias Piipari
2024-11-10 17:59:04 +0000 UTCYeah I am sure, I tried redownloading aswell but every time it connects to server and when I click streamdiffusion, it crashes. I am happy to send you a video from DM if you have free time. Ive already sent you a text
Ada Kanlıbaş
2024-11-05 23:35:52 +0000 UTCHi! have you been able to do this? I have the same problem, with a 4060 I only have 2 or 3 fps the nvidia has 8gb of ram it uses 3gb
maria carame lucenilla
2024-11-04 18:56:47 +0000 UTChey ! which model are you running? sd-turbo is 2.1 based and won't be compatible with 1.5 loras. if not sd-turbo and using a 1.5 model if you can send me links or something to the base SD model as well I can try and recreate.
dotsimulate
2024-11-01 15:18:49 +0000 UTCHey Ada. I am not quite sure. Is there any chance that your project is in the OneDrive location on your pc?
dotsimulate
2024-11-01 15:17:15 +0000 UTCdid any of you succeed with making a LoRA work? Could you please link it to me and tell me your specs? 🙏🏼 thanks
Lorenzo HMZ
2024-11-01 10:44:46 +0000 UTCDoes anyone have issues loading any kind of LoRA? I'm running on a Windows 11, RTX 4080 SUPER, python 3.10.9, cuda 12.1 a SD1.5 model with 3 steps and V2V. Every LoRA has a slightly different type of error; this LoRA here [https://civitai.com/models/109043/skin-and-hands-malefemale-from-polyhedron] should be able to support SD1.5, but I still get a long error starting with this: Error loading LoRA 'polyhedron_new_skin_v1point1': Error(s) in loading state_dict for UNet2DConditionModel: size mismatch for down_blocks.0.attentions.0.proj_in.lora_A.polyhedron_new_skin_v1point1.weight: copying a param with shape torch.Size([128, 320, 1, 1]) from checkpoint, the shape in current model is torch.Size([128, 320]). Anyone that could help?
Lorenzo HMZ
2024-10-31 18:23:14 +0000 UTCI was running streamdiffusion smoothly but fow some reason it just started to crash whenever I start streaming. It connects to the server without problem but for some reason every time I press stream, it crashes and says it is due to touchdesigner/project1/streamdiffusionSD/numphy_share_out. Is it only happening for me, or do I do something wrong???
Ada Kanlıbaş
2024-10-31 15:16:32 +0000 UTCHi Dot i want to ask what is this error, i change pc, try to insall 0.2.3 and this happend " cloning into 'C:/ StreamDiff/StreamDiffusion'... error: RPC failed; curl 92 HTTP/2 stream 5 was not closen cleanly : CANCEL (err 8) error: 6465 bytes of body are still expected fetch-pack: unexpected disconnect while Reading sideband packet fatal: early EOF fatal: fetch-pack:invalid index-pack output "
Julii Haid
2024-10-30 15:24:12 +0000 UTCI'm running streamdiffusion on AWS - is it possible to use that server inside touchdesigner?
Armin Schl
2024-10-27 19:39:20 +0000 UTCWell I tried setting up a new project and had to do tensorrt optimisation again and now it's easily hitting 24fps so I'm happy, also haven't encountered any crashes so pretty happy so far. But I admit I did a bit of a dumb and it seems resolume was ALSO running on the rtx and that was supposed to run on the quadro. But what I get from your answer is that 8 fps is normal for an rtx3090?
RocketShip Trips
2024-10-27 15:55:10 +0000 UTCtensorRT doesnt work with contronet, if you have controlnet active TensorRT will be automatically diabled and youll onoly get the 8fps. Look at the CMD window when its loaded and it will tell you
James Clarke
2024-10-27 14:29:58 +0000 UTCSo I have some performance related issues. Running v2.3 and latest td on RTX3090 and getting only 8ish fps regular and maybe 5fps in v2v/controlnet (dont know if these are normal fps numbers but I think thats pretty low for rtx3090). When I enable tensorrt I got 15fps (although it looked like far more) so great! But ONLY the first time, now every other time after I start it WITH tensortrt enabled I only get 8ish again. Weird thing was, I didn't even close td, I just stopped the stream and started it again later without changing anything. Since then I obviously did reboots and such to make sure that wasn't it, ran it without any other apps open (although when I got higher fps I had a lot of stuff open and running). Should I remove rensorrt and redo the optimizations or just redo the whole streamdiffusion installation? What is the best approach?
RocketShip Trips
2024-10-24 15:33:35 +0000 UTCIs it no longer possible to control the Denoise Schedule of Input 1?
Mirko Ditroia
2024-10-22 17:05:37 +0000 UTCdoes this work for mac?
Jeff Fay
2024-10-20 18:33:52 +0000 UTCHey. If you have troubles with the python window, there is a debug toggle in the settings 2 page that will pause the cmd window on any errors, allowing you to see the issue. Best place to check would be the discord once you know what is going wrong. Check or post to the troubleshoot forum !
dotsimulate
2024-10-19 17:57:14 +0000 UTCseems to be from when installing Tensor getting an error about module streamdiffusion not found
Dan Fallen
2024-10-17 16:05:11 +0000 UTCI'm getting this error "TensorRT not found in the virtual environment" even though I installed it from the operator
Dan Fallen
2024-10-17 16:02:38 +0000 UTCtriton is only on linux version of streamdiffusion, that is not causing your problem
Guglielmo Guarnera
2024-10-17 09:17:53 +0000 UTCHi is there any document on how to download hugging face models and use only the newly downloaded models for StreamDiffusionTD?
giang.it
2024-10-16 10:53:16 +0000 UTCwith StreamDiffusionTD 0.2.3 I'm getting this error: A matching Triton is not available, some optimizations will not be enabled. Error caught was: No module named 'triton' I'm not sure however if this is causing the traceback error that is causing the larger failure which is a show-stopper for me.
Dr. Mindset
2024-10-16 03:16:15 +0000 UTCHello! What’s the best way to troubleshoot? Start Stream brings up a terminal window for half a second before the window closes and status remains Not Streaming.
Cate
2024-10-13 20:42:36 +0000 UTChello, did you figure out how to increase it? I have 16gb too and it shows that currently only 1gb is utilized. thanks!
eclectictasteofbot
2024-10-12 19:27:20 +0000 UTCTrying to get this working on Mac and keep running into this error: Preparing Stream... Process Process-4: Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap self.run() File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/Users/ethan/SD223/StreamDiffusion/streamdiffusionTD/main_sdtd.py", line 369, in image_generation_process pinned_input_buffer = torch.empty((height, width, 3), dtype=torch.uint8, pin_memory=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: Need to provide pin_memory allocator to use pin memory. Please help
Ethan Gates
2024-10-12 12:54:51 +0000 UTCHello, there is an issue with memory allocation if I try to use any other nvidia stuff, like bg removal, stream diffusion will lock out the node, mediapipe gpu will act the same if I try segmentation. I think this might be an issue for the streamdiffusion git repo. I got a 4070ti and the gpu memory is around 6.3/12gb with the tox running an sdxl turbo with control net at 512x512px. cannot create Cuda stream - an illegal memory access encountered.
Dromp Alex
2024-10-11 15:31:09 +0000 UTCHello, everything is working perfectly, thank you very much. However, I have a question: how can I increase the amount of VRAM I am using? I have a 4060 with 16GB, but I am currently only utilizing 4GB to run the Tox
Andres Felipe
2024-10-10 19:50:39 +0000 UTCI Love this update, just hit 56fps on 512x512 SDTurbo with TensorRT Thank you! so Incredible! Please support FLUX.1-schnell soon, this would really take my projects to the next level. and it seems the 4090 has enough headroom for it! Error : KeyError: 'time_embed.0.weight' Failed to load C:\Projects\StreamDiffusionTD-0.2.3\models\Model\FLUX.1-schnell\flux1-schnell.safetensors / SD1.5 + LCM LoRA model from both local and Hugging Face due to: 'time_embed.0.weight'
Sebastain Graf
2024-10-10 01:39:28 +0000 UTChow can I get better fps?, sometimes get 5. fps but after and mostly get 0. etc. fps thanks man
Mario Vishne
2024-10-09 20:37:08 +0000 UTCTY!
Victor E Morales
2024-10-09 18:39:04 +0000 UTCBanger!! Thanks a lot !!!!
virusvjvisuals
2024-10-09 13:51:36 +0000 UTCAmazing. Thank you 🙏
Kamil Nogowski
2024-10-09 07:53:25 +0000 UTCNot fully confirmed to be currently. I should have time to push that soon.
dotsimulate
2024-10-09 06:32:34 +0000 UTCAmazing. Thanks for adding the FPS counter:) cool stuff
Jesse van Aalderen
2024-10-09 06:26:08 +0000 UTCNice! Thank you! :D
Alex Klauck
2024-10-09 06:25:33 +0000 UTCis this Mac compatible?
Victor E Morales
2024-10-09 01:49:57 +0000 UTCWoo hoo! An update that arrives when I actually have time to test it. Thank you for all your hard work!
Kevin Friel
2024-10-09 01:23:10 +0000 UTC
