AI is moving fast in HR. Candidate screening, performance reviews, agentic recruiters that scour LinkedIn and send emails autonomously, these aren't hypothetical use cases. They're live, right now, inside enterprise organizations. And most of them haven't been adequately tested for legal risk.
That was the central message of our most recent Luminos Live, featuring Brenda Leong, Director of ZwillGen's AI Division, and Nick Maietta, our VP of Legal Automation. Together, we walked through three real-world HR use cases, did some issue-spotting, and showed how Luminos automates the testing and documentation that makes this governable.
Here's what we covered.
One of Brenda's core points: compliance and risk are not the same thing. Most AI laws focus on what we'd call "high-risk" use cases -- employment, credit, housing. But the other risk quadrants (reputational harm, shadow AI, visibility risk) create just as much exposure and often fall outside specific AI legislation entirely. A good AI governance program has to cover all of it.
The list of potential liabilities for HR AI is long: discrimination, privacy, security, IP, UDAP, contractual obligations, ADA compliance. The honest answer to "what could go wrong?" is: a lot. GenAI systems generate decisions -- or support decisions -- at massive scale and speed. When something goes wrong, it doesn't go wrong once.
One of the most important and underappreciated issues in AI-assisted HR is where exactly the AI is making a decision, even when it's not making the final decision.
Brenda put it clearly: if an AI candidate screener narrows a pool of 500 applicants to 50, and no human ever reviews the 450 who were excluded, that AI made a decision. It didn't make the hiring decision, but it made the screenout decision with complete reliance. Courts are starting to catch up to this. Laws increasingly use language like "significantly impacted" or "a significant factor in" to capture exactly these situations.
The flip side of "who gets recommended" is "who gets excluded." That false negative is a concrete employment decision, and it needs to be treated as one.
Three Use Cases, Three Different Risk Profiles
We walked through three composite examples drawn from real deployments.
An in-house tool that reviews documents, emails, calendar activity, and Slack messages to auto-generate performance evaluations. The business justification is real: performance reviews are time-consuming and hard to do consistently. The risks are also real. A sample size of 10 for bias testing is statistically meaningless. Employees will game the metrics as soon as they know what's being measured. And if managers are editing from an AI-generated draft without checking it closely, there's no real human in the loop.
Brenda's first question for any client in this situation: Is this actually solving the problem you're trying to solve? Before you get to governance, make sure the system is reliably doing what you think it is.
A GenAI tool that screens applicants, conducts pre-screening conversations, and makes recommendations on who to advance. Demographic data is explicitly excluded from ingestion, which sounds like protection but isn't sufficient. These models are probabilistic. They can learn proxy features that correlate with protected characteristics even when that data isn't directly provided. We've seen it happen. The only way to catch it is to test for it.
Deploying before testing and planning to test after the fact is, as Brenda noted, not just a governance gap. It's potentially negligent. Especially for a system making employment decisions at scale with essentially no prior risk assessment.
An AI agent that identifies candidates, scrapes publicly available information, and reaches out autonomously. Vendor-provided, but that doesn't shift all the risk. As the deployer, you still own the outcomes. There are also emerging questions about FCRA exposure when you're systematically aggregating information to build candidate profiles -- there's active litigation on exactly this question right now.
And the geographic limitations built into the tool? Those might be adding to your bias problem, not reducing it.
Nick demoed the Lighthouse Platform and walked through how we approach this in practice.
The short version: governance has to be operationalized. That means automated testing before deployment (and ongoing after), documentation that captures why design decisions were made, and a memo output that legal and risk leaders can actually review and act on, without needing to dig through raw test data.
We test against what we call "constitutions" -- provisions mapped to specific legal risks like AI transparency, disability screen-out, coded language, and policy compliance. The system ingests input/output data from the AI being evaluated, scores it provision by provision, and generates a memo that captures the full risk picture. When everything passes, approval can be automated. When there are issues, the right stakeholders get notified with the data they need to have an informed conversation.
The goal is to close the gap between what companies think their AI is doing and what it's actually doing, and to make that demonstrable to a regulator, a plaintiff's attorney, or a board.
One point we raise every single time we talk about HR AI, because it keeps getting missed: disability-related screen-out risk.
Metrics like email volume, meeting frequency, and communication style can systematically disadvantage employees and candidates with disabilities -- not because the system is designed to discriminate, but because the benchmarks weren't designed with accessibility in mind. This is a real ADA exposure point, and it needs to be part of every HR AI risk assessment.
The companies that are going to end up in the worst position aren't necessarily the ones with the most aggressive AI deployments. They're the ones who deployed without testing, documented nothing, and are now trying to reconstruct a governance story after the fact.
The good news: this is fixable. The testing infrastructure exists. The documentation can be automated. The legal framework, while complicated, is navigable if you take it seriously.
If you want to see how Luminos approaches this in practice, watch the full recording above. Or reach out directly -- we do this every month.
