On April 12, 2024, Chatwize introduced new subscriptions centered on Bring Your Own Key (BYOK). This feature allows eligible users to set up their own OpenAI API key inside their Chatwize accounts and enjoy unlimited Message Credits (MC). After you
exhaust all MCs inside your Chatwize account, additional AI usage will be charged against your own API key and you will continue to be able to use AI features. New subscriptions do not come with monthly recurring MCs. However, you still have the option to purchase add-ons for monthly recurring MCs directly from Chatwize.

​
Setting up BYOK

To set up your own OpenAI API key, you will first need to apply for one directly from OpenAI’s website. Sign up for an account on OpenAI, then go to this link: https://platform.openai.com/account/api-keys. Additional instructions for obtaining an API key can be found here: https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key. After you obtain your API key, store it in a private and secure location. Chatwize recommends that you use a separate API key for each of your BYOK applications (including Chatwize), and to never share your API key with anyone else. Next, navigate to Chatwize. After logging in, go to the top right corner of the UI. Click the “person” icon the bring up a dropdown menu.
Then, click “Account”. In the “OpenAI API Key” section, paste your API key and click “Add”.
You should be all set.

​
Budgeting for AI usage

In general, using your own API key will be more cost efficient than purchasing MC add-ons directly. To help you estimate costs associated with running your BYOK account, we provide the following references.
OpenAI changes its pricing from time to time, so the information we provide may not always be up-to-date. For latest information on OpenAI’s pricing, please visit https://openai.com/pricing.
The simplest estimate you can use is this:
  • For GPT-3.5 and GPT-3.5-16k, the cost is ~$0.0064 USD / MC
  • For GPT-4 series of LLMs, the cost is ~$0.0028 USD / MC
For example, GPT-4-1106-4k uses 20 MCs per query. To estimate the cost in USD, multiply 0.0028\*20=0.0028 \\\* 20 = 0.056 USD. This is a very rough estimate. Actual spend may be around +/- 20% of the estimated number. This is because OpenAI charges for LLM usage based on “tokens”, and input vs. output tokens cost different amounts even for the same LLM. To understand tokens, we refer you to this article: https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them. There are many parts to a LLM query that Chatwize submits to LLM provider for processing. At a high level, they can be broken down into the parts below: Input:
  • System prompt and metadata
  • User-defined base prompt
  • Variables and definitions
  • Tag definitions
  • Function metadata and descriptions
  • Function parameters
  • Function response
  • Static RAG context
  • Semantic conversation memory
Output:
  • LLM response
  • Response metadata
Chatwize supports a variety of OpenAI LLMs, as well as different versions of the same LLM with custom token limit cutoffs. To help you budget for your usage, we provide a summary based on our default split of reserved input vs. output tokens. Please refer to the following table (all $ are in USD). Please note that this generally represents an upper limit since not all input / output tokens in the reservation window are used every single LLM query.
ModelReserved for InputReserved for OutputCost / Input TokenCost / Output TokenTotal Cost per Query
GPT-3.532008000.00000150.0000020.0064
GPT-3.5-16k152008000.0000030.0000040.0488
GPT-4-1106-1k8002000.000010.000030.014
GPT-4-1106-2k16004000.000010.000030.028
GPT-4-1106-4k32008000.000010.000030.056
GPT-4-0125-8k72008000.000010.000030.096
GPT-4-1106-16k152008000.000010.000030.176
As of April 2024, Chatwize still uses gpt-3.5-turbo-0613 for GPT-3.5. This is because based on our internal benchmarks, this model still significantly outperforms its cheaper counterparts. Other variants like gpt-3.5-turbo-1106-4k and gpt-3.5-turbo-0125 suffer from the “laziness” problem, where generated responses are overly terse and can ignore provided context.
Once we introduce the ability for you to configure your own split of token limit distribution, you will be able to customize the tokens reserved for each category yourself. This means that you will have to calculate your own per-query cost in USD using this formula: 
total cost per query = tokens reserved for input * cost per input token + tokens reserved for output * cost per output token
This formula assumes that all tokens in the reservation window are used, so actual costs may be lower.

​
BYOK for white-label commercial partners

In addition to the costs for MC expenditures during LLM queries, you also need to pay to run our AI multi-agent framework using your own API key. This is independent of whether your users have supplied their API key for their personal accounts. Since the official Chatwize subsidizes its users for all costs associated with running the AI framework, your white-label solution must operate with this premise as well. There are three separate workflows that require your own API key to cover your user’s AI expenditures. The conditions under which they apply are listed below.
  1. AI Agent intent generation
  • Applicable if two or more user-facing Agents are connected
  • Charged whenever a new user-facing AI Agent goes live or an existing one is edited
  1. Query intent classification
  • Applicable if two or more user-facing Agents are connected
  • Charged on a per query basis
  1. Variable extraction
  • Applicable if AI Agent has one or more variables set up
  • Charged on a per query basis
The estimated costs associated with each workflow is as follows:
WorkflowAverage Estimated InputAverage Estimated OutputCost / Input TokenCost / Output TokenEstimated Cost per Run
AI Agent intent generation (gpt-4-1106-preview)6004500.000010.000030.0011
Query intent classification (gpt-3.5-turbo-1106)1000500.0000010.0000020.0195
Variables extraction (gpt-3.5-turbo-1106)10001000.0000010.0000020.0012
All costs from the above workflow will be charged directly against your own API key. This cost cannot be transferred onto your clients.