Janek's Computers

Chatbot worklog

Chatbot requirements and notes

Call 16.08 (beginning of Next.js)

• sem start i beg of oct

• flyers need to go out in 2 weeks

• nothing new, haven’t worked on it since sunday

• we got a lot of speedup through dropping Vue and using this new UI
• bummer it didn’t happen sooner

• next checkpoint is sources

• meetup was very interesting

• money
• wanted to really work on my own project
• freelancing as a way to finance that
• this is inbetween, but doesn’t give me the benefits
• we said let’s write down the requirements, and now I see them I’m a bit scared
• don’t want to leave you with nothing though
 

Meeting 7.04 w/ Wolfi and Jackson

• wolfi will push the repo today

• reverse chatbot:
• ask questions to the user
• based on schematics that we already have
• jackson has a prompt
• first question is “hey, how are you, study?”
• what questions/topics?
• “brandstiftung”
• I will check
• warnings
• hacking
• cost
• premium is separate from bundles
• ai is included in bundle

• strapi
• routes - clear
• controller - make sure to learn and respect the concept
• services - all logic
• maybe I don’t need a single or coll type - maybe just the api
• npx strapi generate - in the backend folder
• don’t use endpoints, use entity API service to get all the data I want
• auth is the same in front and backend
• check the concept of strapi plugins
• admin panel
• content manager
• content-tu
• keep in mind: always create things in singular

• topics:
• admin panel → check errors, but I got an overview anyway
• reverse chatbot
• does the chatbot use any info from our documents?
• interface for gesetze
• algolia search
• semantic search
• i5/dorian

• front is vue2
• nuxt would be good but migration is a lot
• wolfi created a plugin, I can use it to help me fetch data

Refresh, scraping 2, LangChain, GPT-4

Status

• https://heyjura-gesetze.janek.ooo/

• https://heyjura-chat.janek.ooo/
• next step: q&a that refers to gesetze hosted by us
• make chunking and embeddings fit our case
• (if that makes sense - they’re also supposed to overlap etc

• mind algolia and try it out - demo sched + 15k creds

• brainstorming
• (was there a figjam?)
• chatgpt plugins
• reverse chatbot, khan academy inspiration
• panel with columns

Session 2

• brainstorming
• ask “free” AI as well ask
• document types
• gesetze
• court documents / reports
• cases
• “schemata”

• mind a comparison against algolia

• next ste

Session 1

• repo: heyjura-scraping-gesaetze

• progress has been made

• in the process of generating embeddings from {url, text} into a special pkl file

• watch a video about embeddings?

• completed mpaeper tutorial
• pt1: ask a question
• pt2: have a bot

• wider out
• what do we want/can we do?
• semantic search
• question/answer about relevant documents
• alternatives
• algolia, as plugin for CMS
• tutorials and inspo
• other references
• https://platform.openai.com/docs/guides/embeddings/use-cases

Solutions for semantic search + Q

• Content Chatbot by Päpper
• got it to work, but results on his own examples are a bit underwhelming
• made an issue on GH to check if I’m doing something wrong

• paper-qa
• running on a bitcoin paper worked (possibly quite well)
• it’s actually meant for papers (academic) - but maybe that translates to cases?

• https://github.com/jerryjliu/llama_index
• supposed to be similar to paper-qa?
• also used in Irina’s tutorial

• https://github.com/mayooear/gpt4-pdf-chatbot-langchain
• looks good and production ready
• 4k stars and a Chwd tutorial

• official tutorial
• https://platform.openai.com/docs/tutorials/web-qa-embeddings

• check how huberman ai was done

• ChatPDF

• Algolia semser

Resources

Hours & tasks

Meeting notes

• idea: concentrate on one themengebiet

• make a buffer of cases in the database

• idea for prompt engineering: have a two-step process with stories separate to X

• https://learnprompting.org/docs/intro

Fine-tuning report

• Models & usage:
• I fine-tuned 3 times in slightly different ways. We can ignore the first one, and models “16-32-03” and “18-10-11” I included the results and in Playground presets. “18-10-11” should be clearly better, but I’m including both just in case
• for “16”, make sure you keep the arrow (→) and don’t add a space after. For “18”, keep the hashes (###) and empty lines. In general, just change what’s before these special signs

• Results:
• 👜
  Generated cases: results
• as I was already out of time, I’m leaving for you and Vici to judge, but my quick impression was that they unfortunately are not better than prompt eng

• number of examples
• we’re in a bit of a blind spot where we have many more than we can fully utilize in prompt engineering (~4-5) and many less than than are recommended for fully utilizing fine-tuning (~500)
• openai says “doubling the amount of examples leads to linear increase in productiveness”, i.e. with 2x examples we get X % better results and with 4x examples we get 2*X % better results (for example 10% if the first doubling of number of examples resulted in 5% better results)
• the training examples have to be very consistent in format, so I don’t think we can use examples from other books
• it still works, and it’s hard for me to say how much better it would’ve been with 500
• there’s a new but unofficial third way called LangChain. I haven’t had time to evaluate it yet, some good info is here

• next steps?
• it’s possible that I can try generating with solutions. I would just copy-paste and format semi-manually, I think that’d be the fastest. Then I’d generate another fine-tuned model.
• I don’t want to promise anything time-wise, but let’s talk and try
• we can continue brainstorming other uses for AI. not saying this one isn’t good, but I think we can find 2-3 more that are around the same level of good, or maybe at some point better
• there’s a tutorial for “asking questions about a website’s contents”, which could be interesting. In our case the contents would be the gesetze, and people could interactively ask questions about them and hopefully get good answers

Docs for Jackson

Jackson & Janek, before Janek’s leave

• email to Eckbert
• maybe 1st res now

• antrag auf

• for Mon
• OCR software
• screen recording
 

• other AI
• semantic search
• chat
• asking helpful questions that the student has to answer
• answering students’ questions about the law

• results of fine tuning
• no results yet
• timing and money
• pay for N h, and I will finish the rest witout counting time (added in post: was something like 8/10h)
• remember: also try generating the other thing

• what I did so far with

• how does prompt engineering and fine tuning work (I am going to record my screen)
• demo playground, 2 options, temp and other
• learn prompting

• what can be done from my side during your (well deserved) vacation?

• is there a way to easily implement this solution to our website and make it available for registered users?

• (Gesetze scraping)

• (FigJam board)

• (see if I can give you some contacts for funding)
• Dorian Mast - check
• Philippe & Timo?
• Piotrek Grudzień

• lablab ai hackathon

• https://customgpt.ai/

• documentation and übergabe
• give github access to Wolfi and/or Jackson
• video from Jackson
• Notion docs

• our cooperation in the future

Strapi - main CMS

• every content type

• strapi
• routes
• controllers

Inspo for xx

Meeting with the publisher

• nice

1st meeting with Vici

Failure modes

• fall 9 (4a-i-m) ist gut

• logische fehler (z.b. Q gegen P)

• zu viel ähnlichkeit inzwischen

Start on Wed

Tooling

• ChatGPT

• OpenAI Playground

• shareGPT

• 2 other IDEs?

Log

• first prompt: “The following are 11 practice cases for students learning German law. Generate 10 more cases.”

• prompt limitations: 4001 tokens max, where 3800 was the prompt
• seems like we have 18/0.02*1000=900k tokens free

• switching to ChatGPT as a test of whether this limit applies

• ChatGPT 1 - https://sharegpt.com/c/gaikNUd
• a lot of Schadenersatz
• note that compared to GPT-3, you can’t set params like temperature, which would maybe have helped with Schadenersatz

• quick cleanup of cases

• add topics to cases

• 3-with-themen - seems to have

Ideas

• a blind test of cases to see which are AI generated
• maybe in a simple web app
• additionally with a rating of how “good”/useful the case is
• (maybe generate the app itself)

• include the topics of what is being tested, and then ask for cases that test that thing exactly
• you can do a “committee” that selects 8 of 10

• include a “seed” of an idea for a story. This could be “decision made under hypnosis” or “accidental bid on an auction”

• is there any more hidden information, like a specific article that’s being tested?

Rhethoric for the call

• AI progress is exponential, now is the time to attach ourselves to it

• The best is the cooperation of people and AI

Resources

• https://youtu.be/0uQqMxXoNVs?t=1172 - OpenAI CEO predicting AI as a platform akin to mobile, rise of companies that use it

• 2 other Sam Altman interviews

Practice outside of heyJura

• Learnprompting’s tutorial on creating own ChatGPT

• Overment’s book with Shortcuts and Make

• pot. any tutorials that Jackson has sent

Ideas

• heyJura, if successful as a “Layer 3” product could try to position itself as a “Layer 2” company that provides the best AI model for usage by any startup that wants to do something in German law

• argumentation is a good task for ChatGPT

• WolframAlpha ideas are an example of combining “hard” knowledge with an AI element

• components of

Call with Thore and Yannik

• indiv. skripte für Jurastudent:innen

• “click und collect”

• why AI - digitalle bildung zu katapultieren

• fälle

• main customers:
• students
• unis and profs

• limebit (Max Fuchs)

• alexander löser

• fault tolerance

• not that you can replace ppl but make them more effective

AI on legal cases

• google for:
• how to use lms
• for legal

• ask Marc

• ask Grudzień

• ask Piotr Szymański

• ask macOS

• ask LearnPrompting Discord

• email

• brian eno’s oblique strategies

From learnprompting.org

• https://learnprompting.org/docs/basics/standard_prompt
 

Resources

• https://learnprompting.org/

• https://github.com/dair-ai/Prompt-Engineering-Guide

• https://www.lesswrong.com/posts/chevXfQmRYrTZnj8r/conditioning-prompts-and-fine-tuning

• http://www.davidvictorrodriguez.com/uploads/2/6/4/2/26420847/gpt-3_esq_-_evaluating_ai_legal_summaries.pdf
• https://news.ycombinator.com/item?id=26183114

First round of advice from Janek for heyJura

Searching through articles

• in general it’s the kind of problem where you likely want to use a ready-made service. It’s common enough for millions of dollars to have already been put into solutions. It should also end up being pretty cheap or even free before you have a lot of traffic

• in the end I didn’t do research beyond 5 minutes, but from what I knew before, I’d start by checking out Algolia
• It seems like they have a cool option where they’ll talk with you and make you a demo - https://www.algolia.com/demorequest/. I’d definitely try doing that!
• I use their demo app all the time for a news source/forum I read - try it on https://hn.algolia.com. It instantly and with live preview searches 29 million positions. Each is only a few words, so that probably matters - but should still give great results for you
 

Keep AI in mind (GPT-3 and successors)

• it may not be the moment yet, or never, but you should be aware of what’s possible and keep it on your mind for the future

• check out GPT-3 - sorry for not providing resources, but maybe just start with youtube

• one thing I remember it doing it translate a huge paragraph of legal text into a few sentences of simple english. Keep in mind that there will be mistakes, but even things that are only 70% exact can sometimes be useful (maybe makes search better if you don’t have to use the exact words)

• GPT-3 was a revolution, and it’s already 2 years old. GPT-4 might come out late this year or early next year

Shop integration

• modern standard is Shopify, we also use it where I work

• for a simple shop, you could maybe use something simpler and cheaper, but shopify is pretty futureproof

• my suggestion was that you can build a separate shop using shopify’s stuff - I think you don’t need a dev to do that
• then hook it up to shop.heyjura.com and link from the main page
• in the future (or already now) you can integrate it properly into your Vue.JS frontend
• I actually didn’t find an official tutorial for that, but this 3rd party solution looks good: https://vuestorefront.io/shopify
• worth checking out more, I didn’t have the time

Logrocket

• I didn’t look into it properly, so no opinion

• I remember we used https://mixpanel.com at one of my workplaces, I think it’s probably similar, so maybe worth a comparison

Typography for lawyers

• https://typographyforlawyers.com

• a book I thought could be interesting for you. I haven’t read it, but read a bit of the author’s other stuff, and really respect him

Login

• https://auth0.com - biggest and best known. It would be free for a pretty long time, but could get very expensive when you get more users.

• https://firebase.google.com/products/auth - Firebase (owned by Google) is a bunch of different useful services for applications (web and mobile aswell). Could be good if you realize you want more of the different things that they offer, they will probably integrate well with one another.

• https://www.passportjs.org - self-hosted option, meaning it’s a bit more work your side, but free forever. Looks minimalistic, well done, and popular (20k stars on the GitHub repository, active discussions in the Issues section)

• https://supertokens.com - My recommendation. Both self-hosted and hosted-by-them options. Means it’s you host it with them on the free tier (for a long time), and then you can decide if you want to pay or put in more work to self-host and have it free forever. Looks super nicely done, gaining popularity, went through YCombinator (the most prestigious startup incubator in the world). Written in Java (not relevant, fun fact).

• https://supertokens.com/blog/auth-provider-comparison - this is content marketing for SuperTokens, but it’s not too pushy and somewhat objective in my opinion

• https://supertokens.com/pricing - this has a table in the second part two more options I didn’t mention. Again, it’s theirs, but somewhat objective.