• Bay Area Times
  • Posts
  • Google's PaLM 2 reportedly has 340B parameters, vs. 1T of GPT-4

Google's PaLM 2 reportedly has 340B parameters, vs. 1T of GPT-4

Hey, welcome to today's Bay Area Times daily newsletter. If you haven't yet already, follow us on Twitter!

Top stories today:
1. Google's PaLM 2 reportedly has 340B parameters, vs. 1T of GPT-4
2. TikTok is banned in Montana, to be challenged on 1A grounds
3. Google to announce AI tools for advertisers on May 23
4. Stability AI open sources version of DreamStudio, its text-to-image app
5. McCarthy and Biden optimistic on debt ceiling deal
6. Commercial real estate prices fall 1% in Q1, 1st fall in a decade

0. Data and calendar

All values as of 6 AM ET / 3 AM PT, other than S&P500 close (4 PM ET / 1 PM PT).

All times are ET.

1. Google's PaLM 2 reportedly has 340B parameters, vs. 1T of GPT-4

  • Fewer parameters than PaLM, which had 540B parameters. - CNBC

    • “Compute-optimal scaling”: Google had previously disclosed that “this new technique makes PaLM 2 smaller than PaLM, but more efficient with overall better performance, including faster inference, fewer parameters to serve, and a lower serving cost.”

  • PaLM 2 was trained on 3.6T tokens, much higher than 0.78T for PaLM and likely similar to GPT-4 (undisclosed).

Our view: PaLM 2 is likely worse than GPT-4, but Gemini could be better

  • Google released no statistics comparing PaLM 2 with GPT-4, which signals their product is worse. If PaLM 2 had scored better on standardized tests or AI benchmarks, Google likely would have let us know by now.

  • Gemini, the new LLM in development by Google Deepmind (the merger of Google Brain and Deepmind), could surpass GPT-4.

    • OpenAI might be developing GPT-5 in response to Gemini. In April, OpenAI said it was focused on improving GPT-4 and developing an ecosystem around it (i.e., Plugins) before training GPT-5. But things move so fast in AI that OpenAI's plans might have changed significantly. - The Verge

2. TikTok is banned in Montana; ban to be challenged on 1st Amendment grounds

  • In theory, effective Jan. 1, but the law will undoubtedly be challenged in the courts.

  • TikTok says it's unconstitutional: “Gianforte signed a bill that infringes on the First Amendment rights of the people of Montana by unlawfully banning TikTok.”

    • ACLU agrees: “With this ban, Governor Gianforte and the Montana legislature have trampled on the free speech of hundreds of thousands of Montanans who use the app to express themselves, gather information, and run their small business, in the name of anti-Chinese sentiment.”

  • Counters to 1st Amendment arguments include:

    • National security.

    • Separation of conduct and speech, i.e., no particular speech is being banned; the whole platform is. - WSJ

3. Google to announce AI tools for advertisers on May 23

  • According to internal documents, Google's announcements will include the following:

    • Tool for advertisers to generate their own media assets and suggest videos for YouTube creators to make.

    • An internal Stable Diffusion-like product for image creation. - CNBC

  • Meta, the other digital ad giant, last week announced similar advertiser tools, in limited beta, with what it's calling the AI Sandbox.

  • Google Colab, their Python-focused copilot, is also integrating generative AI, positioning itself as potentially a competitor to Microsoft's GitHub Copilot. - Google

4. Stability AI open sources version of DreamStudio, its text-to-image app

Stability AI.

Their strategy is to become the benchmark of open-source AI and sell custom solutions to enterprise

This just-released podcast with CEO Emad Mostaque is very explaining:

5. McCarthy and Biden optimistic on debt ceiling deal, as Dems prepare 2 backups

  • McCarthy: “We finally have a structure to negotiate,” commenting on Biden's openness to cede on something. - CNBC

  • House Dem leader Jeffries signaled support for 2 years of capped spending growth, vs. 10 years of 1% spending growth in the House GOP bill.

  • Biden: “I’m confident that we’ll get the agreement on the budget and that America will not default,” he said before leaving for Japan for the G7 meeting, where Russia and China are the biggest problems. - Bloomberg

    • McCarthy did criticize Biden's trip: “Mr. President, stop hiding, stop traveling.” - Bloomberg 

    • China is meeting the leaders of 5 former Soviet republics for the inaugural China-Central Asia summit in Beijing. - NYT

The Democrats are preparing 2 backups

  1. The 14th Amendment: 5 Senate Dems signed a letter urging Biden to use it to avert the debt ceiling. - WaPost

  2. The House forcing the vote on a clean debt ceiling hike, via the discharge petition mechanism. There's no time for it, though, as even if they had the votes today (all 213 Dems + 5 Republicans), the procedure takes around 25 days until final passage, according to a former House parliamentarian. - WSJ

The Treasury cash balance fell 38% to $87B on Monday

  • Yellen's June 1 deadline could be delayed by a few days, but likely no more than that, unfortunately.

Our view: the GOP is likely to get a deal with Biden, with around half the cuts it originally passed

  • 6 years of capped spending? A fair estimate of what will be agreed on is the mid between the Dem and the GOP proposals: 6 years of capped spending growth, leading to ~112% debt-to-GDP in 10 years.

Japan suggests that debt-to-GDP can still go up a lot higher (226%)

  • Japan's debt-to-GDP is 2.3x higher than the U.S.'s, at 226% debt-to-GDP. Even then, it suffers from a deflation “problem,” not an inflation one.

  • Regardless, what matters most for U.S. economic growth is innovation, i.e., new technologies. What can really accelerate that is not macro but microeconomics, e.g., how long it takes for the FDA to approve a drug.

6. Commercial real estate prices fall 1% in Q1, 1st fall in a decade

  • Moody's Analytics Chief Economist “forecasts prices dropping about 10%, assuming the US skirts a recession. If it doesn’t, the declines could get a lot worse. ‘We’re on a razor’s edge here,’ he said.” - Bloomberg

  • Banks have $700B of commercial real estate loans.

    • $500B is in small banks.

7. Mentions of AI in earnings calls in Q1 higher even than in Q4, when ChatGPT launched

Companies are being very quick in embracing the technology:

8. Other headlines

Tech

  • 61% of Americans believe AI could threaten humanity's future.

  • Shein raises $2B at $66B, down from $100B a year ago; $23B in revenue, expected growth of 40% YoY, and $0.8B in net profit.

  • Netflix's ad tier has 5M MAUs, median age of 34, 6 months after launch.

  • Apple to start manufacturing small micro-LED screens.

  • Deepfake startups received $188M in 2022, from $1M in 2017.

  • Snowflake in talks to acquire Neeva, AI search engine.

  • Varda Space raising $25M pre-C at $500M.

  • Twitter poll: Dems retweet, Republicans reply.

  • Quantum computing: IBM, Google give $150M for university research.

  • Uber to allow teens to ride alone, with parental tracking.

  • Google's .zip and .mov domains are phishing risk: cybersecurity experts.

Biotech

Business

  • Steve Cohen “pretty bullish”; AI could be “as impactful as the internet.”

  • Goldman says AI could boost net margins by 4 p.p. over a decade.

  • Citadel sues ex-employees alleging trade secrets theft.

  • Meta to face a $1B+ fine from Irish regulator for privacy issues

  • India unveils $2B plan (with 5% cashback) to lure in tech factories.

    • Amazon to invest $12.7B in AWS in India.

  • Fox CEO says cable focus is key to profitability.

    • Youtube to launch unskippable 30-second ads on TVs.

  • Schwab seeks to raise $2.5B in bonds, after $40B deposit fall in Q1.

  • Microsoft CMO to employees: stock price “most important lever” for pay raise.

  • Deutsche Bank to pay $75M to settle Jeffrey Epstein suit.

    • Chomsky admits Jeffrey Epstein helped him move $270K.

Crypto

  • Tether to invest 15% of its profits in BTC, already have $1.5B.

  • Voyager expects 36% initial payout by June 1, after judge gives thumbs up.

  • FTX lawyers sue Embed, FTX founders to claw back Embed deal.

  • Ripple buys Swiss custody startup Metaco for $250M.

  • Shaq is still allegedly dodging being served.

U.S. politics

  • Trump knew correct declassification process: National Archives letter.

  • DeSantis to officially announce candidacy next week.

  • Mifepristone: appeals court questions FDA's judgment on abortion pill.

  • George Santos's expulsion effort blocked by House GOP.

  • For-profit schools could lose federal grants if students underperform.

  • Supreme Court refuses to block Illinois law on high-powered rifles.

  • Kaitlan Collins, moderator of Trump town hall, named CNN's 9 PM anchor.

World

  • Russia claims to have “completely destroyed” 5 Patriot missile launchers.

  • Ukraine: Europeans pressure U.S. to give country F-16s fighter jets.

  • U.S. calls Russia to demand release of WSJ reporter and Paul Whelan.

  • Turkey is to blame for censorship, not Elon: Reason.

9. Interesting tweets, memes, and images

Was this newsletter forwarded to you? Sign up here. Liked it? Forward it to friends and get rewards (see below).

What did you think of today's newsletter?

Login or Subscribe to participate in polls.

Thank you for reading Bay Area Times. Got any tips? Email us at [email protected].

Disclaimer: The Bay Area Times is a news publisher. All statements and expressions herein are the sole opinions of the author. The information, tools, and material presented are provided for informational purposes only, are not financial advice, and are not to be used or considered as an offer to buy or sell securities; and the publisher does not guarantee their accuracy or reliability. You should do your own research and consult an independent financial adviser before making any investments. Assets mentioned may be owned by members of the Bay Area Times team.