2024 04 15 llm workflow

integrating llms into my notebook worflows.

im finally giving in and integrating llms into my notebooks. i'll eventually need to experiment with their accessibility. i do believe that llms are assistive technology is a familiar way to interactive computing, one compute rich and the other compute poor. i'll learn to understand this technology as an accessibility tool. it feels like the sharper image selling accessibility to abled people at high markups.

to use on the daily i want this feature to be lazy. i'll start using huggingfae and if i dig this life then ill edit this document accordingly. i want it as a magic and a code fence for midgy. i want caching.

    MAIN = __name__ == "__main__"
    FILE = "__file__" in locals()
    I = MAIN and not FILE
    import functools, asyncio, argparse, shlex, contextlib, io, IPython, textwrap
    from pathlib import Path
    shell = IPython.get_ipython()
    if "initialize" not in locals():
        @functools.lru_cache
        def initialize():
            global langchain, langchain_community
            import langchain.cache, langchain.globals, langchain.chains.conversation.base, langchain.chains.conversation.memory, langchain.prompts.prompt
            import langchain_community.chat_models.huggingface
            langchain.globals.set_llm_cache(langchain.cache.SQLiteCache())
            __import__("dotenv").load_dotenv()

https://python.langchain.com/docs/modules/model_io/chat/chat_model_caching/

create our llm using the default parser values

    @functools.lru_cache
    def get_parser():
        parser = argparse.ArgumentParser(prog="🗩")
        parser.add_argument("repo_id",  type=str, nargs="?", default="mistralai/Mistral-7B-Instruct-v0.2")
        parser.add_argument("-t", "--max_new_tokens",  type=int, default=512*2)
        parser.add_argument("-k", "--top_k",  type=int, default=30)
        parser.add_argument("-r", "--repetition_penalty", type=float, default=1.03)
        return parser

    @functools.lru_cache
    def get_llm(**kwargs):
        with contextlib.redirect_stdout(io.StringIO()):
            return langchain_community.chat_models.huggingface.HuggingFaceEndpoint(**kwargs)

create our llm using the default parser values

@functools.lru_cache
def get_parser():
    parser = argparse.ArgumentParser(prog="🗩")
    parser.add_argument("repo_id",  type=str, nargs="?", default="mistralai/Mistral-7B-Instruct-v0.2")
    parser.add_argument("-t", "--max_new_tokens",  type=int, default=512*2)
    parser.add_argument("-k", "--top_k",  type=int, default=30)
    parser.add_argument("-r", "--repetition_penalty", type=float, default=1.03)
    return parser

ensure that our creditials were accepted.

@functools.lru_cache
def get_llm(**kwargs):
    with contextlib.redirect_stdout(io.StringIO()):
        return langchain_community.chat_models.huggingface.HuggingFaceEndpoint(**kwargs)

%% > activate this to reset the prompt template

i figure it makes sense to curate my own prompt template over time. i'll start with this template and modify the files on disk

https://python.langchain.com/docs/modules/memory/conversational_customization/

>>> assert Path("prompt_template.md").read_text().strip().endswith("them:")

if I:

ipython %%file prompt_template.md The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know. The AI always responds in well structured markdown with headings beginning at level 3. Current conversation: {history} me: {input} them:

establish a conversation chain. we'll want to name these things and curate them over time.

    @functools.lru_cache
    def get_conversation(**kwargs):
        return langchain.chains.conversation.base.ConversationChain(
            prompt=langchain.prompts.prompt.PromptTemplate(input_variables=["history", "input"], template=Path("prompt_template.md").read_text()),
            llm=get_llm(**kwargs),
            memory=langchain.chains.conversation.memory.ConversationBufferMemory(ai_prefix="AI", human_prefix="ME"),
        )

    async def invoke(input, **kwargs):
        global result
        try:
            id = display(IPython.display.HTML(shell.tangle.parser.parser.render(F"pending\n: > {input}")), display_id=True)
            return id
        finally:
            result = await get_conversation(**kwargs).ainvoke(input)
            left, sep, right = result["response"].rpartition("AI:")
            id.update(IPython.display.HTML(shell.tangle.parser.parser.render(textwrap.dedent(right))))

    def chat(line, cell=None):
        initialize()
        if cell:
            asyncio.create_task(invoke(cell, **vars(get_parser().parse_args(shlex.split(line)))))

register the magic functions for chat

    def load_ipython_extension(shell):
        shell.register_magic_function(chat, magic_kind="line_cell")
    def unload_ipython_extension(shell):
        pass
    I and load_ipython_extension(shell)

%% if I:

%%chat
what are the standard ways of representing genomic data?

conclusion

an overall theme is curating your llm over time. the feeling of control and consent is important in the interactive computing experience.

some queries i made making this

if I:

ipython %%chat how do i understand css specificity?

I and await invoke("what is a tagged pdf and how does it relate to an html document? can tagged pdf be made from well formed html?")

I and invoke("do web content accessibiity guidelines apply to pdf? is there a different standard? links are helpful.")

I and invoke("what are the major css apis or concepts like block contexts and 3d rendering?")

I and invoke("could you tell me some more core features of css so that i can better understand ALL of the capabilities")

I and invoke("what are some of the most advanced css concepts a seasoned developer will encounter?")