I’ve been trying out NotebookLM from Google. It enables you to collect a small set of documents that you can then query with AI. This YouTube video does a good job of explaining and promoting the tool:
I’m curious to hear the thoughts of others who are more knowledgeable about note-taking apps than I am. What do you think of this tool?
I’ve been interested but haven’t tried it. How many is a “small number” of documents?
Only 20. I gather a set of sources together using Devonthink with its search facility. You have to use cut-and- paste for articles in Markdown to get them into NotebookLM. For PDF articles, you need to locate the files and then use drag-and-drop.
The premise of the tool is that you can use your own carefully selected sources to have better focus in the query process. So far, I find the tool good at summarizing articles and making suggestions for writing some of the time. However, for many queries, I don’t see an advantage over ChatGPT. The interface makes it easy to save and organize notes along with the results of queries.
I see now the tool is only available to people in the US, like the Vision Pro. That’s unfortunate.
I just spent an hour converting research documents in DT to Google docs and then experimenting with NotebookLM.
The concept has significant potential but the limit of 20 articles limits its usefulness if one is summarizing a lot of research. You also have to import 6 documents a time or the system freezes.
My hope is that Obsidian, DEVONthink, Apple Notes and other apps will have similar abilities integrated within them.
NotebookLM doesn’t suffer fools. I asked it to improve what I just posted, and it came back with:
The source is not clear because there is not a significant thesis or other information to summarize.
I don’t think that will always be a problem: “Gemini 1.5 has an enormous context window, which means it can handle much larger queries and look at much more information at once. That window is a whopping 1 million tokens, compared to 128,000 for OpenAI’s GPT-4 and 32,000 for the current Gemini Pro. Tokens are a tricky metric to understand so Pichai makes it simpler: “It’s about 10 or 11 hours of video, tens of thousands of lines of code.” The context window means you can ask the AI bot about all of that content at once.”
Steven Johnson who consulted on NoteBookLM was a guest on This Week In Google in December and discussed working on the project.
Gemini 1.5 is Google’s next-gen AI model — and it’s already almost ready
Are they customizing a model on the 20 papers, or passing them in as context? (Or both?)
If it’s the latter, 20 PDFs sounds in the ballpark of a 128k context, assuming you’re uploading typical journal publications.
Maybe I’m wishcasting, but I expect that this kind of AI driven functionality will be table stakes in the subscription note taking / PKM / read-it-later space within the next few years.
I worry that it has significant potential to make me lazy! I’ve been test-driving Perplexity and I keep admonishing myself to use it as a way into sources that are new to me or as a refresher on sources I’ve already digested, but not as a substitute for actually doing the work.
Its rate of hallucination of misinterpretation is low - but not zero. When you see a few of its mistakes you will be convinced it is an amazing tool but you have to confirm everything yourself.
I’ve been testing against things I know well enough to spot a hallucination. I haven’t come across one yet, but I did watch it change its answer to a query in real time, which both amused me and gave me pause.
I’d asked it a simple tax question: how to file a return for a deceased person. It spat out a bullet list of steps, then erased it and replaced it with a longer list and some form numbers.
I can’t be sure from the answers it’s giving to my queries. I think it is training on the sources you give it but draws on prior general knowledge to give coherent answers.
I have found a small number of errors - perhaps similar to my own error rate.
Gizmodo feels otherwise however
NotebookLM is said to use “Retrieval-Augmented Generation”.
What is RAG? - Retrieval-Augmented Generation Explained - AWS.
Looks like a cool tool. However, I wonder if some AI does all the summaries etc how much knowledge one gets from this type of second brain into one’s first brain.
I would argue that reading many complex sources, summarising it and synthesising them into something novel is the learning.
However, I see great potential in having something like a personal search engine on your notes. For asking things like: what documents in my repository are related to cloud computing.
That is a legitimate concern and is the concern I have for our students. We are focused on figuring out how to leverage AI without reinforcing intellectual laziness and sloppiness. I suspect we’ll end up modifying our instructional paradigm with more work being done in school, limiting the type of homework assigned, and may even, horror of horrors, require written work be done with pen and paper and then digitized so that students can’t rely on AI to do their work for them. We also plan to expand our computer sincere program to include courses in AI, cybersecurity, along with eSports and Drone Development and Competitions.
To facilitate the review process, I have formed an Advancing Excellence Through Innovation as shown in the screenshot below.
It will be challenging and stimulating to ascertain how best to leverage the positives of new technologies while ameliorating the negatives.
Requiring more worthwhile work to be done in school and less at home is a good instinct, I think, but supervised handwriting and digitising isn’t, IMHO.
As a learner and teacher, I often felt that time in school was too valuable to waste with drudgery. We learn from each other and especially from expert and committed teachers. We do need the ability to read, digest and re-factor written material, of course, and have to learn to do it well, but a room full of silent young people spending hours doing written tasks which mainly reproduce what they already know is a waste. As an adult, I’ve never worked like that, so why would I deny young people the range and variety of ways to do things that I and other mature learners and writers use?
So much of what goes on in schools is driven by the needs of assessment - ensuring that it’s all your own work, that every criterion is met, that a student gets as much credit as possible by doing the most credit worthy things and avoiding any “distraction” that might open up new areas of learning. I know it can’t be avoided but it’s so important to focus on learning and becoming a better learner.
I think the jury is out on AI - I don’t know if relying on AI as a primary source will damage understanding and learning. I suspect it can, and independence of thought relies on having internalised and self-evaluated knowledge and understanding, but critical and evaluated use of any source and especially putting it together with other sources, is very good for independent learning.
Getting students not to be lazy and use technology for the hard part seems to be a Sisyphean task. I am not sure writing by pen and paper will help. Why not generating an essay and copying by hand then digitising?
It is good to see that you and your school try to face these challenges head on.
Steven Levy writing in Wired does see NotebookLM as an assistant:
“I’ve been playing with NotebookLM for a few weeks. The most annoying part of the writing process for me has always been constantly having to leave my manuscript to find the exact information I need in a transcript or document that I want to refer to or quote from. In writing this essay, when I wanted to remind myself of Johnson’s official title, NotebookLM instantly supplied the answer when I requested it.”
I don’t know what you mean. I was referring to requiring research papers, creative writing, and the like to be written in class, not at home, and then digitally archived for grading and records. I was not referring to " spending hours doing written tasks which mainly reproduce what they already know." Their work would be “tutored writing and research time.” This has the benefit of reducing 3-4 hours of homework each night. As a college prep school, our students have rigorous research and writing assignments, service projects, sports, fine arts, and more. That said, we have yet to decide and are still determining if or how this will work, hence, the operations research side of the innovation project. 
So much of what goes on in schools is driven by the needs of assessment - ensuring that it’s all your own work, that every criterion is met, that a student gets as much credit as possible by doing the most credit worthy things and avoiding any “distraction” that might open up new areas of learning. I know it can’t be avoided but it’s so important to focus on learning and becoming a better learner.
That is our focus. We use the term Rigor to define our educational approach. We define rigor as Complexity plus Autonomy, while preserving Relationships and using technology as a tool of Restoration.
As a private school, we are not beholden to the state and have complete control of instruction and assessment. Assessment does not drive teaching; learning goals and learning do.
I think the jury is out on AI - I don’t know if relying on AI as a primary source will damage understanding and learning. I suspect it can, and independence of thought relies on having internalised and self-evaluated knowledge and understanding, but critical and evaluated use of any source and especially putting it together with other sources, is very good for independent learning.
I agree, which is why we are rigorously reviewing our instructional strategies and operational structure (school schedule, homework, assessments, and more).
Regarding AI specifically, our goal is to neither stiff-arm the technology nor abandon students to it. We have yet to determine what that looks like, but we are working on it.
