clients

Simple clients to chat with the models you trained in style.

class PreTrainedClient(style=Style('{} ', None), system='', user='USR', bot='BOT', stop='Stop!', eos_string='\n\n')[source]

Bases: ArgRepr

Chat with a pre-trained model.

Parameters:

  • style (Style, optional.) – A fully configured Style instance, i.e., a particular style to format user input in. Defaults to stripping whitespace characters and appending a space.

  • system (str, optional) – The system prompt to prepend to the conversation. Default top an emtpy string.

  • user (str, optional) – What the user should be called. Defaults to “USR”.

  • bot (str, optional) – What the model should be called. Defaults to “BOT”.

  • stop (str, optional) – A particular user intput that will stop the conversation and terminate the program. Defaults to “Stop!”.

  • eos_string (str, optional) – String that will be attached to model answers if it indicates that it has reached an end-of-sequence. Defaults to two consecutive new-line characters.

Important

Make sure that the eos_string is consistent with how you prepared your corpus and how you set up your tokenizer. It must be tokenized as the end-of-sequence token.

__call__(generate)[source]

Chat with a pretrained model in the given format.

Parameters:

generate (Generator) – A wrapper around your model that accepts a string (the conversation so far) and returns a tuple with the model answer as well as a boolean indicating whether the model answer terminates with an end-of-sequence token or whether the model has more to say.

Returns:

The chat history as a list of dicts with their “role” keys having values user or bot and their “text” values containing the respective text.

Return type:

list

property flat

The conversation flattened into a single string fed to the model.

class Style(template='{} ', strip=None)[source]

Bases: ArgRepr

Format user prompt into a template to encourage model answer styles.

Parameters:

  • template (str, optional) – A python string containing a single pair of curly brackets where the user prompt will go. Defaults to “{} “.

  • strip (str, optional) – Prior to being inserted into the template, the user prompt will be stripped of all word-delimitation characters and, additionally, of all the characters in the given string, both left and right. Defaults to None, which results in only word-delimitation characters being stripped.

__call__(prompt)[source]

Format user prompt by stripping characters and using a template.

Parameters:

prompt (str) – The user prompt.

Returns:

The formatted user prompt.

Return type:

str