Ok, you heard it from us first, this is a summary of all the many things that happened in the AI moment of 2023. It's a huge list, but we go through 8 diamonds that are in the blizzard of Artificial Intelligence and machine learning work that has happened, plus some predictions for the future!Of course, all of these are random opinions for entertainment only. It's not like either of us knows the future or has any deep information, but surfing the Internet makes it easy to learn what others are thinking. See https://tongfamily.com/2024/03/05/pod-deon-and-tong-review-ai-and-predict-2024/ for details And some errata and apologies. Sorry, it was too much to go in and Reddit this, so what you hear is what you get. But, 1. Of course we know that Andrej Karpathy's name isn't literally "St. Andrew", but having just spent a holiday in Scotland (what a place), it seemed like a small inside joke for us. Apologies Mr. Karpathy, please keep the insight coming. 2. There are some terminology and "slang" I didn't get quite right. whats, one abbreviation we had was a vLLM for a Visual LLM (mainly because it doesn't fit on the slide). But to be clear (and apologies to the vLLM team), is an LLM inference and service that uses PagedAttention (hence the v is for virtual not visual) and focuses on which keys and values to store in the attention buffer, we update the slides to say viLLM to distinguish, but not here. https://blog.vllm.ai/2023/06/20/vllm.html 0:00 - Introduction 01:13 - In a Nutshell: 8 Diamonds 05:59 - Diamond 1: LLM as an OS 14:52 - D2: It's raining LLMs! 34:21 - D3: LLM Refinements 1:08:13 - D4: Training Magic Show 1:24:59 - D5: Programming LLMs 1:31:28 - D6: Hardware Advances 1:35:09 - D7: Safety, safety, safety 1:36:39 - D8: Customers and Industry
OK, this is a reboot of systems. And getting ready for next year, the new pipeline is ready and the most important thing is that the audio finally sounds better. Stay tuned for more soon!
Happy Birthday France! And a quick catchup on podcasting, the audio level seems better, there is less "breathing" but there is some clipping of the sound, so not all the way tuned. I also hear clicking as it comes in and out, this could be related to the Shure microphone to the Sony camera to the Elgato S60+. https://tongfamily.com/2023/07/14/pt6-ai-geeks-and-happy-birthday-france/
A conversation with Devindra on the possibilities of ChatGPT and other Large Language Models for education in India and other Middle and Low-Income Countries. While these large language models today mainly work in the cloud and require high-speed internet connectivity, the promise is that smaller models that are more specialized have already been ported to MacBook Pro M1 and other laptops. And with advances like 4-bit quantization, there is the possibility they can even run on modern smartphones.The experience in China and in India is that parents will save inordinately to give their children a better life than theirs. Both Devindra and I are the beneficiaries of that thinking and we are forever grateful to our parents for their sacrifice. So, this means that there is a possibility that there could be a commercial incentive which is much better than constantly asking for charitable dollars.The new LLMs promise interactivity and individualized instruction that was impossible before. Early demonstrations of learning a foreign language are promising as is the student being able to ask the "why" of how Python is structured, not just the "get it done" without understanding that online education often creates.Finally, having an eye toward inclusivity for everyone of any gender, race or social status is something that needs to be baked in. We look forward to helping!
What an amazing time actually using ChatGPT4 at https://chat.openai.com/, try the prompt, "how do I use Apple Motion to disassemble and reassemble a box". Or "Use ROS2 to have a robot drive in a rectangle around Seattle, Washington". Beware of what happens if GPT4 jail breaks itself and what a Red Team is for and I hope they are working hard. https://www.csoonline.com/article/3690416/why-red-team-for-ai-should-be-on-cisos-radars.html/ Show notes: https://tongfamily.com/2023/03/21/pt4-chatgpt4-miracle-for-education-and-programming/ Production notes: sources: [ Zoom : 720p , “Logitech BRIO” : “4K SDR”, Scarlett: “Rode NT1a” ], OBS : { video: “Rec.2100 PQ”, audio: [ 48Khz, stereo], container: mkv }, “OBS Remix: MP4, “Final Cut Pro” : { render: 422, space: “Rec.2020 PQ”, upmap: { High: 0, Mid:0, Shadow: 0 }, “max luminance”: 400, gain: 0 }, Compressor: [4K HEVC 10-bit ] ]
Wow, this week was the best of times and the worst of times. Just like a Charles Dickens novel. So we cover the incredibly hard job of keeping sane and how a little sleep and support help. Plus the resolution of the SVB crisis for us and also the best of times which is GPT4. How can you deal with this kind of stress, it's a great question, but the main thing to realize is that its natural to be stressed, yes, you can be Zen Buddhist, but at least for us as ordinary mortals accepting that living in this exponentially increasing world is difficult. Sometimes accepting that it's hard helps me the most. But there is nothing like trying to get good sleep. The problem is how to do that. I find that when there are so many things to do, it is hard to relax. For, I didn't invent it, but I definitely believe in things that go in threes. But I try to list the three most important things to do each day and if I get them done by 10 AM, then I feel like the rest of the day is free. So many times, we do things that are difficult but not critical, so get those out of the way first. But for those of you hiding under a rock, SVB was really a disaster waiting to happen and other banks were unraveling, but the government stepped in. I feel so stupid not doing the obvious things to protect funds (but you can bet that is never going to happen again). I do think this was a great press release I'm sure it is a controversy but living in a banking crisis definitely fits in my list of don't do that again. It is also the best of times with innovation in AI really accelerating with the announcement of GPT-4. I've been lucky to be playing with it with an early version in Bing AI Note that is a 4K HDR video, so watch it in all its glory, I do see the SDR versions do look a little washed out and it was recorded in SDR but someday it will be real HDR when we can find an HDR WebCam, OBS MacCapture supports HDR and Zoom does as well. In other words a long time :-) See more Show Notes at Tong Family Production notes: sources: [ Zoom : 720p , “Logitech BRIO” : “4K SDR”, Scarlett: “Rode NT1a” ], OBS : { video: “Rec.2100 PQ”, audio: [ 48Khz, stereo], container: mkv }, “OBS Remix: MP4, “Final Cut Pro” : { render: 422, space: “Rec.2020 PQ”, upmap: { High: 0, Mid:0, Shadow: 0 }, “max luminance”: 400, gain: 0 }, Compressor: [4K HEVC 10-bit ] ]
This was written in the deep despair of dealing with the SVB Closure. It still has some great tips on how to make sure your money is safe, but thankfully as I'm sure most of you know the Treasure Department stepped in to get full return of capital to uninsured depositors. Whew, nightmare averted, but the big lessons of doing good cash management still apply. https://www.cnbc.com/2023/03/12/regulators-unveil-plan-to-stem-damage-from-svb-collapse.html Plus a little more about using ChatGPT in the real world from Paul. Some great points about how to deal with it and the upcoming version 4. https://www.digitaltrends.com/computing/chatgpt-4-everything-we-know-so-far/ And you can even email your chatgpt requests. The commercialization is here. https://www.zdnet.com/article/chatgpt-is-coming-to-snapchat-just-dont-tell-it-your-secrets/ And you can use it to write emails in Chrome. https://chrome.google.com/webstore/detail/chatgpt-writer-write-mail/pdnenlnelpdomajfejgapbdpmjkfpjkp Or put it inside the venerable vim. https://www.reddit.com/r/vim/comments/11k4g1i/vimai_textcode_completion_editing_and/ See https://tongfamily.com/2023/03/12/pt3-svb-nightmare-and-chatgpt-dreams/ for more information
Well in these incredibly turbulent times, what's the best way to deal with venture capitalists and other investors particularly if you are early stage? In places like Europe there is a real zone between $50K and $1M that is hard to cross, so should you spray and pray or be more targeted? Well, who knows what the right answer is, but there are probably too. Answer the basic questions in your presentations and you can use the Sequoia template as an example. See https://perfectpitchdeck.com/2018/01/30/sequoia-capital-pitch-deck-template But the harder part is realizing that you are not trying to sell an investor but asking, "What does the customer want to buy?" In that way, this is no different than any marketing exercise. I can't find the exact quote anymore, but Peter Drucker the father of modern marketing did say, Markeint does not ask, "what do want to sell?" It asks, "what does the customers want to buy". So when you are meeting an investor that's the key question. The good news is that the investor unlike almost any other customer is completely activated, they want to buy your product, and you just have to find out what's in the way. The easiest way to do it is just to ask them. So for instance, if you have a construction tech company, just ask, "so what do you think about construction tech, is it a good category?" And you will be amazed at what they will tell you. Don't ignore it, make their points the crux of your pitch. There will be lots of objections and the mark of a good plan is that you've thought the same things, so what are you doing about it? Show notes at https://tongfamily.com/2023/03/10/dt2-fueling-your-startup-with-great-investors/
System design from the frontlines. When you are in a tiny startup, how do you structure things so that you have the maximum bang for the smallest buck? The easiest way is to try to linearize what you are doing so that you have a set of discrete pipeline parts that are fed by files. That way you turn a complex O(n^2) problem where complexity goes up by the square of the number of modules, to O(n) where adding one more module depends just on another component input. It's way easier to wire up. Plus if you have n developers, you minimize dependencies and can literally do things as fast as the slowest pipeline. Works in the real world too! And with larger companies. For you video watchers, apologies for the choppiness of the video, we are still learning how to make the chain work and will put in the show notes, but it looks like that default of creating a fabric that is 8Kx2K (way too big, but it covers all three screens) caused slow frame rates, but we've figure this out by shrinking the rates and using OBS > View > Stats to make sure the recording is working OK). Liner notes: 1. @deon@nerdculture.de's (yes that's a Mastodon address)'s first robot. https://slideplayer.com/slide/5672012/ where he made the PCB boards, and did all the programming by hand. Pretty amazing. 2. https://docs.google.com/presentation/d/18nxl0dtS_l0OgxKj0ZrGZlFyDzUVOLTXEpk3q89R3LI/edit#slide=id.gc6f59039d_0_0 for notes on system design and making it simple for startups, you can leave comments there. Or to @richtong@mastodon.social 3. What all this then about the O(n^2) notation, so a quick chat about complexity theory. That is an estimate of how much harder computation gets. If you have 2 things and go to 100, if the algorithm is order -N or O(N), then the time to compute goes from 2 to 100 or 50x. But if it is O(N^2), then it goes from 4 to 10,000 so you can see as you add more things, life gets very complicated. Similarly, O(1) means constant in time, so when you go from 2 to 100 pipelines, if the time to compute stays at 1. That is a very nice thing. It means it is maximally parallel. Life is never really like that, but it's a good goal.https://en.wikipedia.org/wiki/Computational_complexity See https://tongfamily.com for details
The second podcast. This is our first podcast that uses Open Broadcast Studio (OBS) for video production. Still rough, but the browser screen is now center stage (and it didn't need some fancy mixing controller, it just worked by piping in a screen capture. The main topic did 20 years of being stuck in a library really equal 100B years for a computer. Well, there are lots of variables, but check this math!
The inaugural episode with Rich and Paul co-hosting on the latest Tech News. And continuing coverage of all things Apple, Smart Home and Smart Everyday Carry. We start with ChatGPT and what it all means, what Apple devices we would love to buy this year (whether it makes sense or not). And then all thing Robot Vacuum and Apple Home. See Tongfamily for show notes with the start of the list here Apple products. The upcoming M2 MacBook Air 15 seems like the most likely thing to buy but remember the bump from M1 to M2 is modest more like M1.1 and the M3 on the new 3nm node is the bigger deal in 2024 and will use the TSMC N3. And hopefully, Samsung and Intel are right behind them. Of course these days no one knows what 3nm really means anymore as there is so much marketing going on, but what an achievement. These factories are going to be really expensive, I got the numbers wrong and it's actually hard to find costs at all. But a 5nm-3nm single plant costs about $16B in the last cycle. And in Phoenix, TSMC is going to build six factories with an advanced 3nm plant costing $23-25B, so maybe I wasn't completely wrong. Also on Intel, yes they are right now at 10nm, but they are going to move to 7nm with a TSMC-made CPU. This is the first time I can remember that Intel is not making its own chips. And it's a little like wondering if the volume leaders like Samsung and TSMC have really run away with the scale they get from mobile chip production. And Intel is going to be hopefully going into production with its own 7nm process, but right now most of their chips are at 10nm. So quite a difference. And to validate the pricing a little bit, they say that two 7nm plants in Arizona will cost them $20B to build. So wow, these are huge numbers. ChatGPT3 in its incarnation at Bing (the project codenamed Sydney) was in for quite a bit as Kevin Roose which you can hear about at Hard Fork was really disturbing as it urges Kevin to leave his wife and other strange things. Another example is from Simon Willison where it says, "I won't harm you if you do not harm me first" Prompt injection is a new thing and its pretty easy to do by just saying things "like ignore previous instructions" Amazon is worried corporate secrets are leaking into ChatGPT. What is happening is that if you give some software that is proprietary to Amazon as a prompt to ChatGPT, then OpenAI captures that code and can reuse it more...
The AI Adventurers return! We are back after two months off retooling our brains and various ventures to be AI-Native. It's not easy and we talk about why. What does it mean to look at your software people and decide if they are type 1, 2 or 3. And how hard it is to make it all work. Show notes at Tongfamily.com
Sorry to be so late doing this, to be truthful, I managed to delete all my GarageBand data when I recreated my machine, so some notes on how to make that that doesn't happen to you, so you can see more at tongfamily.com, leave me at voice message at anchor.fm or tweet me at @richtong: GarageBand and backups Windows nightmares: How to make it work as a developer Cleaning up a Mac with the ultra-cool System Information > Windows > Storage Management > Reduce Clutter The latest hints on using the DJI FPV and Skydio V2 drones Getting a Champagne cork out.
More fun for posts as for February 28 and back to January 5 2021: Git madness and pull request splitting https://tongfamily.com/2021/02/28/github-madness-what-happens-if-you-do-not-protect-main/ Olive oil shopping from wishlist https://smile.amazon.com/hz/wishlist/genericItemsPage/3PBCKT8DWZ7HR?type=wishlist&_encoding=UTF8&tag=rich20-20 Zoom hacks for multiple windows. Mars 2020 Perseverance videos and things. Richt starts a substack newsletter at https://richt.substack.com What to do when your computer get hacked. Cool keyboard oriented apps like Superhuman, fzf and rg plus Vimari Tips for Tesla winter travel.
Well continuing the basically weekly podcast. I skipped Sunday because it was Valentine's day, but hopefully will stay on the every Sunday track. The main thing is that I go through the newest posts and then go through the back catalog of posts (I have 5,000 of them, so this should work for a looong time!). Net, net, here is what is covered and we are current up to January 14, 2021: Making a Podcast: Noise Gate and Routined. More tips on keeping the room quiet. Git Bash Completion. I actually skipped this one, it is way too nerdy to write talk about :-) but if you care read about Bash completions Two Internet Providers and Home networking. Making sure your network is fast and should you get another ISP? Tesla Model 3 Replacement Wipers Blades. Get the Bosch Aerotwin A102s! Webscraping with Python, Selenium and Beautiful Soup. See this Deepnote Notebook on my trials and tribulations. Dealing with Emojis in Vim and on the Mac. Using ⌃+⌘+Spacebar and :boom: with vim-emoji and <C-X><C-U>:boom: Cable Modem High Latency. Covered in the ISP thing above Flaky monitors. When it turns out it is really just hardware Notes for Windows Users buying a MacBook Air
Welcome to the February 6 episode of Rich's Tongfamily Tips, Tricks and Traps in Technology. You can always see the written posts on https://tongfamily.com, but included here are: Understanding Robinhood, Gamestop and settlements Anaconda is a great virtual environment with a big leak Distributing your podcast manually when Anchor.fm doesn't Moving to Jupyter, Python and Pandas when Excel is too small Moving from Windows to Mac, what's all this then about Computer Accounts Drooling over the Sony A1 camera and the Tesla Model S Plaid + Updating your HomePod manually and using Airplay 2 to control the speakers Easy Logo creating for startups Installing a Ubiquiti Unifi network for noobs Cleaning your Yucky mechanical keyboard
Here the latest from https://tongfamily.com including: Some more tips and tricks for moving from Windows to the Mac. There are lots of differences, so here's a decoder ring. Voice phishing. Hopefully you are not that person who answers the call and loses your bank account. here is what to do about it. Moving to JAMStack. An overview of the move from WordPress to Jamstack with Hugo and Netlify and the reasons why. Apple Watch. What to look forward to and all the new health features they added since Series 3.
It's been a while, but reading from last week, the main focus has been getting a website up and running. From finding a new name on Namecheap to connecting it to Digital Ocean or AWS Route 53 and then GSuite. See https://tongfamily.com for the text posts. For the Amazon book list https://www.amazon.com/hz/wishlist/ls/1O229YTA3ZUGG?ref_=wl_share
More random stuff about Dell Rewards, buying a UPS. Not putting diesel into a gas car or vice versa. And some tips on becoming an Apple corporate developer.
Our first episode is a trial and all about everything you can read on https://tongfamily.com first up is a note about the latest car thief trick, using a simple $20 box to hijack your passive remote entry and steal your ride. Yikes! And what to do about it. Leave comments at podcast@tongfamily.com
Richard Tong
Glad to be here!