AI for ESG: Private vs. Public Markets

AI is reshaping ESG (Environmental, Social, and Governance) analysis in both private and public markets, but the approach differs due to data availability and regulatory frameworks.

• Public markets: Benefit from structured, standardised disclosures (e.g., CSRD, TCFD). AI is used for tasks like real-time monitoring, sentiment analysis, and identifying greenwashing. However, the abundance of data can lead to challenges like high noise-to-signal ratios.
• Private markets: Face fragmented, unstructured data (e.g., contracts, local news). AI helps streamline due diligence, extracting insights from limited information. Yet, inconsistent reporting and data gaps remain hurdles.
• Regulations: Public markets operate under stricter rules, demanding transparent AI systems. Private markets enjoy flexibility but must address concerns around transparency and comparability.
• AI tools: Techniques like natural language processing (NLP), knowledge graphs, and anomaly detection are employed differently across markets. Public markets focus on broad monitoring, while private markets prioritise deep-dive analysis.

AI's role in ESG is evolving, shifting from static assessments to continuous risk monitoring. For private firms, this means tackling fragmented data with tailored tools, while public firms leverage structured data for compliance and insights.

Quick Comparison:

Both markets require tailored AI strategies to address their unique challenges and maximise ESG integration.

Data Availability in Private vs Public Markets

Private and public markets differ significantly when it comes to data accessibility and structure. Public markets rely on mandatory filings and standardised sustainability reports, which provide structured data that AI can easily process. On the other hand, private markets deal with customised documents like PDFs, emails, loan notices, and credit agreements. These unstructured formats make extracting and analysing data much more complex ¹.

This contrast shapes how AI is deployed. In public markets, AI can work with standardised datasets and perform tasks like sentiment analysis on continuous disclosures. Meanwhile, private markets face challenges due to limited and fragmented data, creating a clear divide between the two.

Data Scarcity in Private Markets

Private markets often struggle with data availability. Unlike public companies, private firms are not universally required to disclose ESG (Environmental, Social, and Governance) metrics, leaving significant gaps. Information on climate risk, diversity and inclusion (DEI), and supply chain responsibility is often missing or scattered across various documents ⁶. Even when such data is present, it is frequently self-reported and lacks the independent verification processes common in public markets.

Another issue is information asymmetry. Limited Partners (LPs) often depend on General Partners (GPs) for data, which is curated but not always transparent ⁷. AI steps in to streamline this process, reviewing extensive documentation and cutting down due diligence timelines from weeks to hours ². However, as Aileen Sartor, ESG Product Manager at Holtara, points out:

AI can only support the collection of ESG data that is stored in documents, which may not be the case for companies with immature ESG approaches, especially smaller companies ⁵.

Despite these hurdles, there is progress. 75% of Private Equity signatories now assess ESG materiality at the portfolio level, signalling a shift towards more structured data practices ⁶. Standardised frameworks like the UNPRI PC-PE ESG factor map and the ILPA ESG assessment framework are helping to organise data in a way that AI can process, though adoption remains uneven across the sector.

Data Richness in Public Markets

Public markets, on the other hand, benefit from continuous disclosures through frameworks like SASB, GRI, and TCFD. This creates a wealth of data, allowing AI to benchmark performance, track trends, and spot anomalies across thousands of filings. The transparency and structure of public market data enable advanced analyses that private markets often cannot achieve.

However, this abundance of data is not without its challenges. Greenwashing - where companies exaggerate their ESG credentials - remains a significant issue. AI plays a critical role here by cross-referencing sustainability claims with external sources such as media reports, NGO evaluations, and operational data. Bryan Dougherty, CTO at Arcesium, highlights this limitation:

AI doesn't solve foundational operational issues: it magnifies them. Firms see enormous automation potential, yet in practice, they're finding that AI can also amplify weaknesses in data quality ¹.

While public markets provide a rich source of data, this doesn't guarantee accuracy. Instead, it gives AI more material to analyse for inconsistencies and potential misrepresentation. The transparency of public markets supports robust AI-driven insights but also exposes the ongoing challenge of detecting greenwashing within existing regulatory frameworks.

sbb-itb-6ca8558

Regulatory Requirements Across Markets

Beyond data challenges, regulatory requirements play a major role in shaping how AI is used in ESG risk analysis. Public markets operate under mandatory frameworks like the EU's CSRD and the UK's SECR, which contrast sharply with the more flexible, tailored AI approaches seen in private markets that rely on non-standardised disclosures ⁸¹⁰¹². These differences directly influence how AI processes ESG data in these distinct environments.

In private markets, firms face fewer mandatory disclosure rules, which allows them to create bespoke AI frameworks tailored to specific industries and internal needs ¹⁰¹². As the CFA Institute highlights:

private companies do not face the same disclosure requirements as their public counterparts, so data is harder to come by ¹².

However, the gap between private and public markets is shrinking. Institutional investors are driving this shift, with private market ESG funds raising £92 billion in 2022 - more than triple the £29 billion raised in 2020 ¹². At the same time, AI-related disclosures in corporate filings surged from about 4% in 2020 to 43% by 2024, reflecting heightened regulatory focus across both markets ⁹.

Lighter Oversight in Private Markets

In private markets, regulatory flexibility allows firms to develop tailored AI tools suited to their specific needs ¹⁰. This has given rise to "Private AI" - closed systems trained on proprietary data, which ensure enterprise-level security while supporting customised ESG frameworks ¹⁰.

For instance, in December 2025, Apax Partners introduced "ApaxGPT", an AI tool designed to analyse Confidential Information Memorandums and draft Investment Committee memos ¹³. Similarly, Blackstone launched "Norm Ai" in 2025, a tool that processes marketing materials to meet regulatory requirements while safeguarding sensitive data ¹³.

However, this flexibility comes with challenges. Blackstone's Chief Technology Officer, John Stecher, stresses:

AI adoption in private markets isn't about chasing hype. It's about building the right data foundation first, then applying AI where it can truly move the needle ¹³.

There’s also the issue of balancing customisation and accuracy. Nicolai Wadstrom, Partner at Ares, highlights:

For regulated industries, there has been slower adoption as most of these [AI] solutions are more probabilistic... Deterministic AI solutions are available, but this requires a deep understanding of what are fairly complex systems ¹³.

While lighter oversight allows for tailored ESG approaches and proprietary data use, it also raises concerns about transparency and comparability ¹². It can also lead to "AI-washing", where firms make exaggerated claims about their AI capabilities. A stark example of this occurred in March 2024, when the SEC fined Delphia (USA) Inc. and Global Predictions Inc. a combined £400,000 for false AI-related statements ¹⁴. Former SEC Chair Gary Gensler issued a clear warning:

If a human is using AI to defraud investors, they will 'likely be hearing from the SEC' ¹⁴.

Strict Compliance in Public Markets

Public markets, in contrast, operate under stricter regulatory demands. The EU AI Act, set to become enforceable in August 2026, is the first comprehensive AI regulatory framework ⁹. Such regulations push public market firms to adopt standardised AI tools capable of handling continuous disclosures under frameworks like TCFD, CSRD, and ISSB.

This regulatory pressure creates both opportunities and challenges. Public market AI systems must process vast amounts of structured data while adhering to evolving standards. For example, in early 2025, Meta Platforms raised nearly £60 billion to expand its AI data centres, using a dual-market approach that included a £30 billion public bond issuance alongside private placements ¹¹.

Unlike private markets, public firms cannot rely on proprietary "black box" algorithms. Instead, they need AI solutions that are transparent, auditable, and capable of meeting regulatory scrutiny. Deterministic AI systems, which provide verifiable and reliable outcomes, are becoming essential ¹³.

The lines between private and public markets are increasingly blurred. While private markets enjoy more flexibility, institutional investors are demanding the same level of ESG integration seen in public markets. Pauline Thomson, Head of Data Science at Ardian, underscores the stakes:

AI is going to be a huge differentiator and a big risk for firms that don't take it seriously. Those firms will be destroying value without even realising it ¹³.

AI Techniques: Private vs Public Markets

AI techniques differ significantly between private and public markets due to the contrasting nature of their data environments. Public markets benefit from structured and standardised disclosures, such as annual reports, CSRD filings, and earnings transcripts. AI tools in these markets rely on techniques like sentiment analysis and algorithmic screening to ensure compliance and uncover instances of greenwashing ⁴. In contrast, private markets are characterised by limited transparency, requiring AI to extract insights from a variety of unstructured sources, including contracts, local news, and social media. This is achieved through tools like natural language processing (NLP) and knowledge graphs ¹⁵.

These differences in data availability and regulatory requirements have driven the evolution of AI techniques tailored to each market. In private markets, AI must interpret industry-specific jargon and convert it into standardised risk categories - a task that traditional methods often struggle to handle ¹⁵. Dr. Benjamin Krusche, Strategy Director at Clarity AI, highlights the shift towards dynamic data analysis:

Relying on once-a-year collected data points from private companies will look very old-fashioned. The future is dynamic: static data overlaid with an intelligence layer that tracks real-world events ².

Private markets particularly benefit from AI's ability to streamline due diligence, reducing what used to take weeks into just a few hours. Meanwhile, public markets leverage AI for real-time monitoring of global news and market signals, enabling companies to react promptly to changing conditions and sentiment ³. However, both environments face challenges. Private markets contend with scarce and inconsistent data, while public markets must navigate high noise-to-signal ratios and corporate greenwashing ³⁴.

AI Methodology Comparison Table

The growing emphasis on continuous monitoring further sets apart AI applications in private and public markets. By 2024, 64% of deal teams reported using AI in at least one diligence process, with systems capable of analysing over 10,000 documents and more than 50 types of reports simultaneously ³. This shift from static, point-in-time analysis to ongoing risk tracking aligns with the longer investment horizons in private markets and the regulatory demands of public markets ².

These tailored approaches highlight the need for custom AI solutions to address the unique challenges of ESG analysis in private and public markets.

AI Solutions for Private Market ESG Analysis

Private market firms often grapple with a unique problem: making sense of fragmented ESG data while ensuring compliance and maintaining audit-ready documentation. AI platforms tackle this challenge by converting complex credit agreements and sustainability reports into standardised, comparable metrics. This is achieved using natural language processing and structured frameworks, as seen with platforms like Axion Lab ¹⁸¹.

Axion Lab offers AI-powered tools specifically designed for private market stakeholders. These tools facilitate analysis across various areas, including sustainability, operational due diligence, and legal and commercial assessments. Security is a top priority, with features like end-to-end encryption, GDPR compliance, and zero data retention, ensuring sensitive information remains protected ²¹.

A key feature of effective AI solutions for private markets is traceability. Unlike generic chatbots that can provide unsourced or inaccurate responses, advanced platforms link their insights directly to source documents, complete with page numbers and excerpts ²¹. Sergei Maslennikov, Co-founder of Axion Lab, highlights this importance:

Decision-making today is defined by efficiency, coherence, and right use of information ²¹.

This level of transparency allows investment teams to verify ESG risk flags instantly, meeting both internal governance needs and the expectations of limited partners (LPs). By offering this source-level clarity, these platforms not only enhance decision-making but also adhere to strict regulatory requirements.

AI solutions also bring efficiency to ESG analysis. Automated document reviews can cut due diligence time by up to 70%, with platforms processing and validating KPIs up to 10 times faster than traditional manual methods ³. This transition from weeks-long manual reviews to hours-long automated analysis enables early detection of ESG risks during the sourcing and origination stages. This means managers can evaluate how sustainability factors might impact potential returns well before investments are finalised ³²⁰.

To address evolving regulations, leading AI tools streamline the process by collecting ESG data once and mapping it across multiple frameworks such as SFDR, TCFD, and CSRD. This eliminates redundant work while maintaining consistency ¹⁹. Combined with automated audit trails that document every conclusion, these systems turn ESG analysis from a compliance task into a strategic advantage ³.

Addressing Limitations in Both Markets

ESG analysis faces challenges in both public and private markets. Public markets benefit from structured disclosures but often fail to catch emerging controversies as they unfold. On the other hand, private markets may provide more detailed operational insights but lack consistent reporting standards. To tackle these issues, hybrid AI strategies combine various techniques to address these market-specific gaps.

Natural language processing (NLP) plays a critical role by extracting ESG signals from unstructured data sources like social media, local news, and niche industry reports - areas often ignored by traditional methods ¹⁵. Knowledge graphs enhance this further by mapping relationships between entities, helping to trace how controversies might spread through subsidiaries or supply chains ¹⁵. In fact, AI analysis of NGO complaints has been shown to identify material ESG risks up to eight months earlier than conventional approaches ²².

Anomaly detection is another powerful tool, applying consistent criteria to flag irregularities in financial and operational data that human reviewers might miss due to fatigue. For instance, it can uncover deceptive practices like social-washing, which often evade traditional scrutiny ²². Christopher Wright, Head of ESG Risk Monitoring at Norges Bank Investment Management, highlights the importance of AI in this context:

AI is particularly powerful for assessing ESG risk because so much of the underlying information is inherently qualitative ¹⁷.

Building on these techniques, predictive analytics provides forward-looking insights. By forecasting signals such as customer churn, margin stability, and KPI volatility, it enables firms to evaluate management projections before making significant investments ³. Additionally, tools like computer vision and satellite imagery offer concrete evidence of impacts, such as deforestation or methane leaks ⁴²³. By 2024, 64% of deal teams reported using AI in at least one part of their due diligence process, with workflows cutting document review times by an average of 70% ³.

However, the success of these AI-driven methods depends heavily on data quality. As Bryan Dougherty ¹ points out, AI can amplify existing issues with poor-quality data. To ensure reliable insights, firms must standardise their internal processes and clean their financial and operational data. A combination of automated data extraction and human oversight - where experts focus on interpreting high-stakes scenarios while AI handles repetitive tasks - creates a balanced and effective framework for ESG risk management across both public and private markets ³⁵. This integration of human expertise with automated systems is key to navigating the complexities of ESG analysis.

Conclusion

The contrast between private and public markets in ESG analysis highlights a key challenge: data availability and oversight. Public markets enjoy standardised disclosures and extensive historical records, enabling AI to focus on tasks like real-time sentiment tracking and controversy monitoring across a wide array of companies. On the other hand, private markets - representing around £17.5 trillion in assets under management as of September 2025 ¹⁶ - struggle with limited transparency. For instance, disclosure rates for Scope 1 emissions in certain regions are as low as 23% ²⁴. These differences demand tailored AI strategies for each market.

In public markets, AI processes large volumes of disclosed data efficiently. However, in private markets, the approach is quite different. Here, AI must work with fragmented documents, estimate missing metrics, and navigate multilingual sources to uncover hidden risks. As Dr Benjamin Krusche, Strategy Director at Clarity AI, aptly puts it:

Relying on once‐a‐year collected data points from private companies will look very old-fashioned ².

The dissatisfaction with current AI tools is evident - only 6% of private equity firms reported being fully satisfied with their AI solutions in 2025 ²⁵. The way forward lies in blending automated data extraction with human expertise. AI can handle repetitive, data-heavy tasks, while human analysts focus on interpreting complex, high-stakes scenarios. For private market firms seeking to address these challenges, platforms like Axion Lab offer AI-powered due diligence tools. These tools enable early and actionable insights across sustainability, operational, and digital domains, turning limited data into a strategic advantage.

As frameworks like SFDR and CSRD continue to drive regulatory changes, the need for scalable and advanced AI solutions becomes increasingly urgent. While the technology to address these challenges exists, the real hurdle is implementing it at scale.