Tokenizer for Claude models #177
Description
The current way of trimming strings and messages are quite hacky, partly because we don't have access to Claude's tokenizer (we do for OAI models). Once we find good tokenizers for claude models, we will need to revisit the trimming functions.