Fine-Tuned Models
What are they?
- A new model that was trained by taking a foundation model (e.g. GPT-4o) and giving it additional training data (typically in the form of a list of sample prompts and expected outputs) - this model can then be used like a foundation model via the API platform
- The model’s user and system prompts can be set via the API, just like any other model - none of the prompt is baked in - all the training data goes into the actual model weights/parameters
- Fine-tuning is offered by all major model providers - OpenAI, Anthropic and Gemini
- Designed to help adapt general purpose models to make them capable of solving more specific problems
Caveats:
- Requires the user to cultivate an extensive library of labeled training data
- Fine-tuning to a specific set of examples can actually reduce performance on things outside of the training set
- Typically no longer recommended for most use-cases, particularly in an automation context, outside of very specialized use-cases
Current State of Support in Relay.app:
- Customers can use fine-tuned openAI models trained by them on their linked platform accounts.
Custom GPTs
What are they?
- This is a feature offered by OpenAI specifically
- It consists of a base foundation model (typically GPT-4o in OpenAI as of this writing) with:
- A customized system prompt
- Access to tool-calling with a pre-selected handful of natively supported tools that can be optionally enabled:
- File search (RAG) with pre-uploaded files
- Code-execution
- Web browsing
- Configured and available to ChatGPT Plus subscribers to build their own, published in a marketplace in ChatGPT
Caveats:
- Not capable of using fine-tuned models - can only use base foundation models
- Not linked to any OpenAI platform account - and thus inaccessible via the API and to Relay.app (or other automation products)
- Unfortunately, Relay.app (nor other automation products) cannot call custom GPTs, as they are not exposed in the OpenAI API platform
Current State of Support in Relay.app:
Not feasible - as per the above, due to the lack of API support from OpenAI we’re not able to support calling custom GPTs from Relay.app. We will be evaluating future API improvements that may change this over time.
Assistants
What are they?
An OpenAI-specific API that replicates the functionality of Custom GPTs, and makes them available available on the OpenAI platform
Caveats:
- Almost all of the same caveats and capabilities as Custom GPTs - doesn't let you use fine-tuned models - they are based on foundation models
- Assistants are not visible in ChatGPT / plus - assistants are purely OpenAI platform constructs
- This also means they cannot be published in the Custom GPT marketplace
Current State of Support in Relay.app:
Assistants support is on our roadmap. If using an Assistant is a key use-case for your needs, please reach out to support@relay.app and we would be happy to discuss specifics.
Concluding thoughts:
- We have observed a lot of confusion around the terms and what they mean in our customer-base
- We do not recommend these constructs for automation use-cases
- We believe that most of the time, users will be able to solve their automation use-cases with simple AI steps with well-engineered prompts. Saving prompts to be re-used across workflows is easier done through our Sequences feature that allows segments of workflows to be re-used.
