Here's what caught my eye in the world of AI this week:

Reforge’s AI Extension really caught my eye this week. This is part of their broader path to bring expert guidance directly into the tools we use daily. Reforge has been my go-to source for product content for years, and I cannot recommend them highly enough. The amount of high-quality and free content they provide is off the charts. They also offer excellent training courses and a premium membership that’s worth every penny. This is also an excellent example of a content company taking there niche IP and spinning up a Generative AI product. One way to go about this is to make it accessible in products that already have big subscriber bases. Prediction: I think niche content rich companies will be an area to watch for acquisitions.

1. Enhance Your Jira Experience with Reforge's AI Extension

Bit: Expert feedback on your Jira tickets/projects just got easier with Reforge's AI Extension integration in Atlassian Jira as well as Notion, Coda and Google Docs.

Bytes:

Seamless Integration:

• Direct Feedback: Get expert feedback on your specific Jira tickets and projects without leaving the platform.

• Improved Collaboration: Facilitates better collaboration between PMs and engineering teams by addressing key project questions.

Key Features:

• Contextual Insights: Provides the necessary context to implement tasks correctly.

• Success Tracking: Offers guidance on the tracking needed to report on project success.

• Stakeholder Updates: Identifies stakeholders to update when projects go live.

• Rollout Strategies: Recommends rollout strategies, whether all at once or gradually.

User-Friendly Design:

• Non-Intrusive: The integration is collapsible, ensuring it’s not too intrusive while still being highly accessible.

• Avoid Pitfalls: Helps PMs, engineers, analysts, and designers avoid common pitfalls in software development.

Outcome:

• Enhanced Productivity: Direct access to expert advice within Jira enhances team productivity and project success.

• Daily Tool Integration: Brings Reforge experts' guidance directly into your daily tools for continuous improvement.

For more details, check out their AI Extension here. Check out Daniel Wolchonok short and informative demo.

Absolutely loved the insights from the AIQCON Conference this week, especially the talk on “From Predictive to Generative: Uber’s Journey” given by Kai Wang and Raajay Viswanathan.

2. Uber’s Journey: From Predictive to Generative AI

Bit: Dive into Uber’s impressive seven-year journey in building an End-to-End MLOps platform.

Bytes:

In-Depth Journey:

• From Predictive to Generative: Explore Uber’s transition and advancements in AI through their platform, Michelangelo.

• Industry-Leading Features: Michelangelo's integration of cutting-edge tools like Horovod and Palette feature store sets it apart in the evolving ML landscape.

• Case Study : Uber’s Machine Learning Platform “Michelangelo”

• Demetrios Brinkman the organizer of the AIQCON conference, hosts a fantastic video podcast, check it out.

  • Video Summary: Uber's Michelangelo platform has evolved significantly through three major phases, enhancing its capabilities from basic ML predictions to sophisticated uses in deep learning and generative AI. Initially, Michelangelo 1.0 faced several challenges such as a lack of deep learning support and inadequate project tiering. To address these issues, Michelangelo 2.0 and subsequently 3.0 introduced improvements like support for Pytorch, enhanced model training, and integration of new technologies like Nvidia’s Triton and Kubernetes. The platform now includes advanced features such as a Genai gateway, robust compliance guardrails, and a system for monitoring model performance to streamline and secure AI operations at Uber.

Expert Insights:

• Streamlined ML Development: Min Cai’s from Uber talk highlights how Michelangelo enhances ML developer experiences with a seamless UI and a unified API framework.

• Future Vision: Uber envisions Michelangelo as an open ML platform, integrating top-tier third-party and in-house components for enhanced flexibility. Can’t wait!

• Most companies lack the expertise and time to develop such a sophisticated system. But here's the good news! Check out Byte 2 below.

The good news is that there are plenty of players in the LLMOPS space, so there's no need to spend seven years building this solution in-house. Remember when CI/CD was cutting edge? It's not a matter of "if" but "when" you'll fall behind if you're not actively implementing LLMOPS. While it's easy to put one model into production and maintain it, scaling requires automation. Additionally, ensuring auditability will be extremely difficult without uniform processes and patterns. The race is on!

3. Magic Quadrant for Data Science and Machine Learning Platforms

Bit: Gartner's 2024 Magic Quadrant evaluates the leading platforms for data science and machine learning, highlighting key players and their strengths and cautions.

Bytes:

Strategic Insights:

• Future Roles: By 2027, half of data analysts will become data scientists, shifting the focus to AI engineering.

• Platform Capabilities: These platforms integrate code-based libraries and low-code tools, supporting data scientists and business counterparts through the data science life cycle.

Key Features:

• Comprehensive Toolsets: Platforms support a range of users to develop predictive and prescriptive analytical techniques across various data types.

• Unified Workflows: Enable the creation of consistent and reproducible models, enhancing collaboration and productivity across teams.

Market Dynamics:

• AI Democratization: Platforms are evolving to support low-code and natural language interfaces, allowing domain experts to create predictive models.

• Enterprise Adoption: The market is driven by demand for AI solutions, with hyperscalers gaining traction but opportunities remaining for diverse platform offerings.

Outcome:

• Enhanced Productivity: Reduced cycle times and entry barriers for creating predictive models, enabling widespread AI adoption.

• Strategic Selection: Insights from the Magic Quadrant guide data and analytics leaders in choosing the right DSML platform for their needs.

• Want Deeper LLMOPS Dives? Check out our past Newsletters

  • The Future of AI Prompt Engineering: Dead or Evolving?

  • 8 bits for a Byte - Your AI Weekend Curated Reads: 6-21-24

I recently had a conversation with a reader who was asking about AI in Finance and Accounting. Marc Andrusko and Seema Amble from Andreessen Horowitz had a great post on LinkedIn on this topic.

Key Take Away For Now: What LLMs lack (for now!) is the ability to do complex calculations and quantitative analyses — two skills crucial to the accounting profession.

4. Death, Taxes, and AI: How Generative AI Will Change Accounting

Bit: Generative AI is revolutionizing the accounting industry, bringing efficiency and transforming traditional workflows.

Bytes:

Transformative Impact:

• Efficiency Boost: Generative AI automates repetitive tasks in bookkeeping, tax preparation, and auditing, saving time and resources.

• Labor Market Shift: With 75% of CPAs retiring within the next decade and fewer new entrants, AI addresses the growing demand for accounting services.

Key Applications:

• Data Collection and Ingestion: AI-powered tools can automatically pull data from various sources, simplifying reconciliation and error checking.

• Research and Analysis: LLMs streamline research by summarizing information and answering complex queries based on vast datasets.

• Report Generation: AI generates audit-ready reports and checklists in a firm's distinctive style, enhancing accuracy and compliance.

• Client Service and Advice: AI delivers high-quality insights, allowing accountants to provide ongoing, optimized advice to clients.

Strategic Considerations:

• Incentive Alignment: Different buyer personas, such as tax departments with hourly rates versus auditors with fixed fees, influence the adoption of AI.

• Talent Development: While AI augments junior staff, it is essential for firms to continue training programs that expose new accountants to comprehensive workflows.

Outcome:

• Scalability and Flexibility: Firms can handle increasing client demand efficiently, offering high-value advisory services with AI's support.

• Future Readiness: Accounting firms adopting generative AI position themselves as market leaders, prepared for the evolving landscape.

A former JPMC co-worker, good friend and Product Thought Leader Luciana Ma, recently wrote an outstanding post that breaks down AI from a Product perspective in which she does a fantastic job of explaining AI Agents and their applicable business use cases.

5. AI Agents from a product perspective

AI Agents: Revolutionizing Industries with Advanced Intelligence

Bit: Discover how AI Agents are transforming industries by leveraging natural language processing (NLP) to understand queries, generate responses, and interact effectively.

Bytes:

Understanding AI Agents:

• Versatile Functionality: Unlike LLMs and RAGs, AI Agents create personalized budgets, monitor account balances, recommend investments, and manage bill payments.

• Key Capabilities: AI Agents excel in tool-use, reasoning, memory, and self-reflection, enabling them to perform complex tasks and improve over time.

Key Features:

• Tool-Use: AI agents utilize APIs to interact with external systems for data retrieval and action execution.

• Reasoning: They employ logical processes to interpret data, make decisions, and predict outcomes.

• Memory: Storing information allows these agents to learn from past experiences and enhance their performance.

• Self-Reflection: Advanced AI agents analyze their actions and refine their strategies for continuous improvement.

Industry Applications:

• Coding: Tools like GitHub Copilot and DeepCode automate coding tasks, boosting efficiency.

• Customer Service: AI agents like Ada and Intercom streamline customer support by resolving queries autonomously.

• Data Analytics: Platforms like Hex and Sisu Data use AI to provide deep insights and enhance decision-making.

• Sales & Marketing: AI tools such as Drift and Clari optimize customer engagement and sales strategies.

• Financial Services: Solutions like Kasisto and Zest AI automate financial tasks, from fraud detection to payment reconciliation.

• Human Resources: AI platforms like Pymetrics and HireVue enhance recruitment and talent management processes.

• Legal Services: Tools like Luminance and LawGeex automate legal document review and compliance.

• Education: AI solutions like Duolingo and Gradescope support personalized learning experiences.

Innovative Solutions:

• Real-World Applications: AI agents address long-tail problems, such as YouTube’s content moderation, by detecting and flagging inappropriate content.

• B2B vs. B2C: AI-Agent products are primarily B2B due to reliability requirements and consumer preference for human interaction.

Strategic Insights:

• Synthetic Data: Startups can leverage synthetic data to overcome the lack of high-quality training data and enhance AI models.

• Integration Opportunities: Companies like Otter AI demonstrate the potential for integrating AI across platforms to create seamless experiences.

Outcome:

• Enhanced Efficiency: AI agents automate routine tasks, allowing human workers to focus on higher-value services.

• Industry Transformation: By integrating advanced reasoning, memory, and self-reflection, AI agents are poised to bridge the gap between human potential and machine intelligence.

Explore the exciting advancements and opportunities AI agents bring to various industries, reshaping the way we interact with technology and driving innovative solutions by taking a deeper dive into Luciana Ma’s substack post.

6. Friday Funnies 🤣 .

• The question is how long will it take? What do you think?

Send me your best AI meme and I will post the Top 5 and put it to a vote by our readership! I am sure anyone can do better! Winner gets a bottle Moet Champagne! Have to get at least five memes for there to be a competition.

Andrew Ng and Nikita Namjoshi making the world a better place.

7. Carbon Aware Computing for GenAI Developers: Reduce Your AI's Carbon Footprint

Bit: As machine learning models grow in size, their carbon footprint increases. It's crucial to quantify and mitigate these emissions as AI scales.

Bytes:

• Carbon Awareness: Learn to quantify and reduce the carbon footprint of your AI projects.

• Expert Guidance: Google Cloud Developer Advocate Nikita Namjoshi teaches practical techniques for sustainable AI development.

Key Learning Points:

• Real-Time Carbon Data: Query the ElectricityMaps API for up-to-date information on regional electricity grid carbon intensity.

• Eco-Friendly Routing: Route model training jobs to data centers powered by low-carbon sources like wind, solar, hydro, and nuclear energy.

• Carbon Measurement: Use Google Cloud's Carbon Footprint tool to measure emissions from ML training, inference, storage, and API usage.

• Optimal Scheduling: Optimize training schedules to run when clean energy is most abundant in a specific region.

Outcome:

• Sustainable AI Practices: Develop and implement strategies to minimize the environmental impact of your AI models.

• Enhanced Efficiency: Improve the efficiency and sustainability of your machine learning operations.

Enroll in their free short course, "Carbon Aware Computing for GenAI Developers," to make a positive impact on the environment while advancing your AI skills.

Key Take Away: The study underscores the potential of finetuning with synthetic data to enhance LLM performance on long-context tasks. Synthetic data is becoming increasingly crucial for training and improving models, offering a valuable resource for startups that often lack access to high-quality data for model training. Keep an eye on this area, as it will be pivotal for future advancements in AI.

8. From Artificial Needles to Real Haystacks: Improving Retrieval Capabilities in LLMs by Finetuning on Synthetic Data

Researchers: Zheyang Xiong, Vasilis Papageorgiou, Kangwook Lee, Dimitris Papailiopoulos

Enhancing LLMs for Long-Context Tasks

Bit: Recent studies reveal that Large Language Models (LLMs) often struggle with accurate information retrieval and reasoning over long contexts. A synthetic data finetuning approach can address these limitations.

Bytes:

Finetuning Approach:

• Synthetic Dataset: A carefully designed synthetic dataset was used with numerical key-value retrieval tasks to enhance LLM performance.

• Experimental Success: Models like GPT-3.5 Turbo and Mistral 7B showed significant improvements in information retrieval and reasoning after fine tuning on this dataset.

Key Findings:

• Skill Transfer: Finetuned models and demonstrated a 10.5% improvement in real task evaluations (e.g., MDQA at position 10 for GPT-3.5 Turbo).

• Consistent Performance: While general benchmark performance remained stable, models finetuned on synthetic data avoided the hallucination issues seen with other long-context augmentation data.

Strategic Insights:

• Synthetic to Real: Their analysis shows effective transfer of skills from synthetic data to real-world tasks.

• Stable Benchmarks: Models like Mistral 7B finetuned on their data maintained stable performance on general benchmarks like TriviaQA, unlike those finetuned on other datasets.

Outcome:

• Improved Retrieval and Reasoning: Finetuning LLMs on synthetic datasets significantly enhances their ability to handle long-context inputs without compromising general performance.

• Reduced Hallucination: Their approach mitigates the risk of hallucination, ensuring more reliable AI outputs.

Explore the full potential of finetuning on synthetic data to boost the capabilities of LLMs in handling long-context tasks effectively. I will write more on this topic in future Newsletters.

AI & ChatGPT Mini Crash Course - Eliminate workplace burnout & save 16+ hours every week. Learn 20+ AI tools, prompting techniques & hacks for free.

Save your seat here (first 100 people only) ⏰

👇 If you like my newsletter I think you will also like 👇