TIPO and DanTagGen : Power your Image Prompting

Are you searching for the best LLM that can optimize your prompts? Here it is. TIPO and DanTagGen are the LLMs(large language models) that help you to "text-pre-sampling" in the workflow. It generates the detailed tags relevant to your inputted prompts by adding extra elements. 

It's dependent on KGen library. Basically, KGen is a Python tool extracts portions of code from a larger Fortran program and converts them into standalone, verifiable, and executable kernels. This tool is particularly useful for testing, debugging, and optimizing Fortran code.

Now it's possible to use these to your Stable diffusion WebUI like Automatic1111/Forge and ComfyUI.

Table of Contents:

Installation in ComfyUI:

1. First you need to have ComfyUI installed on your machine.

2. Update ComfyUI from Manager by choosing "Update ComfyUI".

3. Move to "ComfyUI/custom_nodes" folder. Navigate to the folder address bar and type "cmd" to open the command prompt. 

Clone the repository by typing the following command:

git clone https://github.com/KohakuBlueleaf/z-tipo-extension.git

Now, you also need to clone another repository i.e. ComfyUI custom script by pythongosssss. Use need these commands:

git clone https://github.com/pythongosssss/ComfyUI-Custom-Scripts.git

If you already have ComfyUI-Custom-Scripts installed, then just update it. Simply move to "ComfyUI/custom_nodes/ComfyUI-Custom-Scripts" folder. Open your command prompt using "cmd" on the folder address bar. Then type "git pull" into the command prompt to update the custom nodes.

4. Restart ComfyUI to take effect.

5. Get the workflow from your "ComfyUI/custom_nodes/z-tipo-extension/examples" folder. Drag and drop to ComfyUI.

As usual, load your related model to the load checkpoint node. This is actually not dependent to the workflow. Our main goal is to optimize the images with accurate and detailed prompting.  

Use ban tag to input what you don't want into your art generation like watermark, nsfw etc. 

6. Select the LLM model you want to put into from the TIPO node. If you want you can work with the TIPO node to add to any other custom workflow, then you can find under "utils/promptgen" category. To get the actual configuration, you can move to the detailed configuration provided below.

Installation in Automatic1111/Forge:

1. You need to have Automatic1111 or Forge installed.

2. Install it from the Extensions tab of  WebUI. Search its name from the "Available" tab. If not found, then alternatively select "Install from URL". Put the URL by copying and pasting provided below:

https://github.com/KohakuBlueleaf/z-tipo-extension.git

Now, click the "Install" button to initiate installation.

3. After installation, just restart your WebUI by clicking on the "Restart UI" button.

4. You will see a new nl prompt text box under the positive prompt box and a tipo accordion beside that.

Valid Settings:

You should use the recommended configuration to get optimized results.

Parameter	Description
Tag Length	Very Short: 6~17 tags Short: around 18~35 tags Long: around 36~53 tags Very Long: around 54~72 tags Recommended: Short or Long
NL Length	Very Short: 1~2 sentences Short: 2~4 sentences Long: 4~6 sentences Very Long: 6~8 sentences Recommended: Short or Long
Ban Tags	The blacklist of tags you don't want to see in the final prompt. Regex supported.
Prompt Format	The format of the final prompt. The default value is recommended format of Kohaku XL Zeta model: <|special|>: 1girl, 1boy, 1other, ... <|characters|>: character tags <|copyrights|>: copyright tags (series name) <|artist|>: artist tags <|general|>: general tags <|meta|>: highres/lowres/absurdres are all meta tags <|quality|>: score_xxx, masterpiece, best quality... <|rating|>: safe, sensitive, nsfw, explicit... <|generated|>: generated NL prompt <|extended|>: extended NL prompt (If nl prompt is not provided, extended will be filled with generated NL caption. If both extended and generated are in the format but nl prompt is not provided, TIPO will generate 2 NL prompt.)
Seed	The seed of prompt generator. Since we use temperature/top k/top p sampling, so it is not deterministic unless you use the same seed. -1 for random seed.
Upsampling Timing	After: After other prompt processing, for example: after dynamic prompts/wildcard. Before: Before other prompt processing.
Temperature	Higher means more dynamic results, and Lower means better coherence between the tags.

Conclusion:

TIPO and DanTagGen LLMs give you the ultimate experience of optimizing techniques to power your prompting to the next level. It can be loaded into any Stable Diffusion WebUI like ComfyUI, Automatic1111, and Forge.