Enhancing Your AI Artwork with Civitai’s Extensive Tools and Resources

Civitai is a website that contains various fine-tuned models, LoRAs, and other resources for Stable Diffusion.

When incorporated into your workflow, these resources can dramatically increase the quality of your outputs. Here’s everything you should know about Civitai:

What Civitai has to Offer

Civitai is a multi-faceted platform that offers the following:

• Stable Diffusion Resources: Includes models, LoRAs, Upscalers, VAEs, Poses, Embeddings, Motion modules, Wildcards, and Workflows. These are intended to be used with one of the many user interfaces available for Stable Diffusion including WebUI, ComfyUI, Focus, etc.
• Image Generation: Just like Clipdrop, Mage.Space, Midjourney, Civitai allows you to generate images right on the site. This is a great way to test out the models before you download them and use them in your own creations.
• Model Training: In their “Train a Model” section, you can upload a dataset of images to then generate a LoRA. This allows you to easily create images in a specific style or theme.
• Articles: Users can share their workflows and insights into how their understanding of Stable Diffusion has evolved over time. Worth reading if you want to level up your Stable Diffusion skills.
• Bounties: This section allows you to post requests to the community in the form of Buzz (Civitai currency) to have someone create a LoRA at your request. This is good for those uncertain about training and would rather have someone else do it.
• Image & Video Hosting: You can create images on site with the many models available or you can also share your own creations with the community. Short videos (120 seconds max) can also be hosted on the site.

All About Models on Civitai

As of today, December 19, 2023, there are 7,359 models available on the site. This is a massive amount of models and can be overwhelming to navigate – especially for someone just getting started with Stable Diffusion.

We’ll try to break this down to be more approachable.

Model Types

First and foremost, it’s important to know that nearly all the models available on Civitai are derivatives of a Stable Diffusion base model. This largely includes any one of the following: 1.4, 1.5, 2.0, 2.1, XL, XL Turbo, or Video Diffusion.

You can see this available in their filtering panel:

The models available are fine-tuned versions of these base models, each trained on specific datasets to serve different purposes and generate diverse types of outputs including photorealistic portraits, anime, cartoons, 3D renderings, and more.

Understanding the Model Card

When clicking on any ‘Checkpoint’ you’ll see some important information right away:

1. Version Release History & Variants: Includes access to all previous versions of the model, allowing you to choose an iteration that might suit your needs better. The author of the model may also include a VAE Fix (often in XL models), inpainting capabilities, or other enhancements that differentiate one version from another.
2. Details: Includes information such as whether it is a checkpoint or LoRA, when it was last uploaded, and what base model it was built upon. The base model, in particular, is important as it will inform you at which image resolution and aspect ratio the model operates best, as well as giving insight into the original training data and techniques used to construct the model.
3. Author Notes: For popular models, in particular, the author will commonly share how to get the best results from the model. This may include information on the prompting style, best sampling method, steps required, CFG scale, and the type of images you should use for optimal outputs. Author notes are invaluable as they provide the user with a unique insight into the creator’s intentions and expert tips that can help to maximize the model’s potential.
4. Suggested Resources: While the models often will be able to provide adequate outputs by themselves, the author may have also made embeddings (textual inversions) or LoRAs that will add additional layers of detail to the generated images. This may provide more cinematic lighting effects, improved texture detail, or even stylistic coherence that matches certain artistic styles.
5. Discussion & Gallery: Lastly, users of the models may have questions or share outputs they’ve created, which is where the discussion and gallery sections come into play. Here, users can showcase their creations, exchange tips, and provide feedback to one another, fostering a community of practice around the model.

Copying Prompts Effectively

For most models, the author will share the prompt, negative prompt, and other parameters that were used to generate specific results. This allows you to copy the information and generate an identical image on your own machine.

The shortcut to copy a prompt is easy, but not at all obvious.

Here’s what you should do:

Look for the ‘i’ icon in the bottom right corner of an image. Then click on the copy generation data button:

Once copied, open up Automatic 1111 WebUI, select the same model from the dropdown, and then paste the information into the prompt:

After pasting, click the ↙️ icon to populate all the necessary fields. Once done, just click generate to reproduce the image on your machine.

Installing Models

There are a few ways you can add models to your local Stable Diffusion program.

• Download the Model: The first is to simply download it from the Civitai site and place it into the respective directory. For example, when using WebUI, you would place it into the stable-diffusion-webui/models/Stable-diffusion directory.
• Civitai Link: If you are a supporter of the site, you can gain access to the Civitai Link integration, allowing you to quickly add models from the site to your local machine. You can see a short video of this being done here:

• Program Extensions: If you are using the Automatic1111 WebUI, you can install the Civitai+ Browser Extension. This allows you to search for models right from the WebUI interface and install them with a single click. Super nice and helps to streamline your workflow.

Here’s what it looks like:

• API Method: Lastly, Civitai offers an API you can freely access to download resources. While it may be overkill for a single model, it can be useful if you are looking to download a large number of models at once or are renting a cloud instance and want to automate the process of downloading models. You can learn more about the API here.

LoRAs

LoRAs, also known as Low-Rank Adaptation, can extend your creations to take on a different style or design.

Much of the information we shared earlier about models carries over to LoRAs. You can find them on the same platforms, and they often come with the same types of metadata and author notes to help you integrate them effectively.

Working with LoRAs from Civitai

When adding LoRAs to your prompts, the dataset on which they were trained varies greatly in quality. And as such, when downloading LoRAs, it is essential to review the user feedback along with the optimal weights.

For example, when working with the SDXL Frosted LoRA, you would want to set the weight to anywhere between 0.7 and 1 – so something like <lora:SDXLFrosted:0.7> or <lora:SDXLFrosted:1>.

Additionally, many LoRAs have a trigger word associated with them to evoke the desired style. In the example of this LoRA, the trigger word is ‘FrostedStyle’. You can see all trigger words in the model card when downloading the LoRA:

Ultimately your prompt will look similar to this: “<lora:SDXLFrosted:0.7>, FrostedStyle, beautiful high-quality digital art of a transparent glass snowman, glowing within with multicolored lights, festival, extremely high quality”

Size of LoRAs

Unlike fine-tuned models that carry a total size > 4.0GB, LoRAs are relatively small and will be roughly 100MB in size, making them a more manageable addition to your toolkit. Their smaller size is due to the fact that LoRAs only contain the parameters necessary to adapt a pre-existing model rather than replace it entirely.

This means that the core AI remains the same, but it is ‘steered’ in a new direction by the LoRA, much like a lens focuses light to alter the image seen through it.

Training LoRAs

For those that don’t have a GPU at their disposal, Civitai allows you to train LoRAs right on the site. The cost is about 500 buzz per LoRA – and can be used to create any sort of character, style, or concept.

Civitai has a good Knowledge Base article on their site for LoRA Training along with some general guidelines when it comes to your dataset as well.

Civitai stresses throughout that good captioning is an important part of model training and ultimately generating good outputs.

As the saying goes, “garbage in, garbage out,” meaning that the quality of the output is directly affected by the quality of the input.

If you want to learn more, this thread on Reddit has some good insights.

Image Creation

Civitai also allows you to effectively test out any of the models or LoRAs on the site to generate images.

All you have to do is hit the ‘Create’ button on the top right of the site or within a model card (checkpoints or LoRAs) and you’ll be able to generate an image:

Currently, you can queue up to 10 at a time, so you can test different models, prompts, and other parameters to see what works best for your needs.

Right now it is free, however, as stated in their official documentation, upscaling and advanced plugins will require Buzz in the future:

Articles

In the spirit of open source, you document your creations, workflow, and other insights on the site. This is a great way to share your creations with the community and help others learn from your experiences.

A few standout articles include:

• How to Use the IP-Adapter for Video Animation
• Camera Framing Angles
• Textual Inversion & Training Guide

Poses

ControlNet is a powerful way to control the pose of your generated images. It allows you to control the position of the head, arms, legs, and more. Within the ControlNet, you can use different control types including canny, OpenPose, Depth, and others.

Civitai has numerous pose packs available to download. These organized resources allow you to manipulate the outputs of your images to create more dynamic and interesting compositions.

For example, you could import a pose of people fighting, doing workout routines, or even just standing around. This can be a great way to add more depth to your creations.