Allow using Azure OpenAI

[UmerHA]

Allows using Azure OpenAI instead of OpenAI (solves #325)

gpt-engineer [project_path] --azure --engine [your_azureopenai_model_name]

To be consistent with Azure OpenAI terminology, the model is called an engine instead.

Requires to set AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_KEY as environment variables.

[UmerHA]

@herbutant because you opened #325: Could you test the pr for me? I don't have azure openai access. thanks!

[herbutant]

@UmerHA , thanks for this.

I made one small change locally to also pull the api_version from ENV because this can also vary (e.g. in my case it is "2023-03-15-preview"). With that it runs like expected.

if not os.getenv("AZURE_OPENAI_VERSION"):
raise ValueError("To use Azure OpenAI version please set a "
"AZURE_OPENAI_VERSION enviroment variable.")

openai.api_version = os.getenv("AZURE_OPENAI_VERSION")

Should an amendment to the README.md with Azure specific instructions also be part of the pr?

[UmerHA]

@herbutant agree - have adjusted the readme & made AZURE_OPENAI_VERSION an optional env variables that defaults to 2023-05-15.

[AntonOsika]

Would be great to create a separate function handling this to keep it tiy.

[AntonOsika]

I realize that if we want to support this we should in fact have a separate object:

The LLM model.

I'd be happy to merge if you find a good way to "abstract away" the LLM object!

[AntonOsika]

I think we need to create a more general abstraction for LLMs if we want to support azure to not make the code messy!

Let me know if you have thoughts

[UmerHA]

@AntonOsika Agree, I think we have 2 options:

1. write a thin wrapper around LangChain's abstraction (BaseChatModel for chat models / BaseLLM for non-chat models)
2. create our own simplified abstraction

I would vote for using LangChain, as LLM integration is not this project's purpose and LangChain does that well already. I'd be happy to implement the wrapper, as I'm a LC contributor as well. :)

If you think option 2 is better, I propose the following abstraction:

class AI with methods:

• factory
• response
• update_token_count

subclasses of AI implement these methods:

• __init__
• _response (called by response)
• get_token_usage

---

Details:
Option 2 in Pseudocode:

class AI
	self.model
	self.token_usage_logs = []

	@classmethod
	def create(cls, model_family_name, model_name, ai_config_filename):
		# read ai_config_filename
		# create a new instance with type depending on model_family_name (e.g. new OpenAI, or new AzureOpenAI)
		# 	and pass params from ai_config_filename to it

	@abstractmethod
	def __init__(self, model_name, ai_config_filename, **kwargs):
		# determine model_name or use default (eg gpt-3.5-turbo for OpenAI)
		# load configs (eg api key) from kwargs or from env
		# initialize model (eg  validate configs)
		
	@abstractmethod
	def _response(self, messages, prompt=None):
		# get response & meta-data (eg token_usage)

	@abstractmethod
	def get_token_usage(self, meta_data):
		# get & return token usage (handle both streaming & non-streaming cases)

	def response(self, messages, prompt=None):  # corresponds to current `next` method
		# logger.debug("starting response")
		
		# ai_response_object, meta_data = self._response(messages, prompt=None)
		
		# if is_generator(ai_response_object):  # streaming
		#	stream & print reponse chunks
		# 	ai_response = join(chat)
		# else:  # not streaming
		#	ai_response = ai_response_object[...]
		# 	print(ai_response)
		
		# logger.debug("response completed")
		# 	messages += [{"role": "assistant", "content": "".join(chat)}]
		
		# meta_data['answer'] = ai_response
		# self.update_token_count(self.get_token_usage(meta_data))
		
	def update_token_count
		# udate token usage log

then the OpenAI / AzureOpenAI would look like this:

class OpenAI(AI):
	def __init__():
		# determine model_name or use default (eg gpt-3.5-turbo for OpenAI)
		# initialize model (eg ensure all configs are given)
		
	@override
	def _response(self, messages, prompt=None):
		# get response & meta-data (ie token_usage)

	@abstractmethod
	def get_token_usage(self, meta_data):
		# get & return token usage (handle both streaming & non-streaming cases)

and

class AzureOpenAI(OpenAI):
	def __init__():
		# determine model_name
		# initialize model (eg ensure all configs are given)
		
	@override
	def _response(self, messages, prompt=None):
		# get response & meta-data (ie token_usage)

	# get_token_usage(self, meta_data) is same as in OpenAI

This would work for all kinds of models (chat/non-chat, streaming/non-streaming).

Let me know!

[AntonOsika]

Hey appreciate the thinking on this!

Based on discussions in other PRs it looks like we will use langchain abstractions.

Because of that and that we have conflicts I’m closing this for now — feel free to open.