Calculating Chargebacks for Business Units/Projects Utilizing a Shared Azure OpenAI Instance

Jayendran 

As a rule of thumb is that one token generally corresponds to ~4 characters of text for common English text. This translates to roughly ¾ of a word (so 100 tokens ~= 75 words).

You can use libraries like tiktoken (Python), or gpt-3-encoder (javascript)  for most GPT-3 models to programmatically fetch tokens for Azure OpenAI response for streaming calls.

You can find some more libraries here: openai-cookbook/examples/How_to_count_tokens_with_tiktoken.ipynb at main · openai/openai-cookbook (github.com). In the example above, you can easily add an extra call to any of these tokenizer libraries to get the tokens count and return the response back to APIM.

P.S. Microsoft makes no endorsements or guarantees of third-party libraries.