How the PO/BA Role Changes in Synthetic Data Projects for AI Training

I want to share some reflections on working on projects where teams create SDG (Synthetic Data Generation) datasets for AI training. From my experience, the role of a Product Owner or Business Analyst in this space changes so much that the classical approach to writing requirements becomes fundamentally different. The PO/BA is still involved in defining scope, requirements, and acceptance criteria, but the way this is done is completely different.

Why is SDG used?

In the classical sense, SDG is needed when real data does not exist, is very limited, or is extremely difficult to obtain. For example, validating LiDAR systems for cars in scenarios where a person suddenly runs onto the road. Or training drones to recognize camouflaged tanks. In the first case, it is not exactly humane :) In the second, there is simply too little real-world data.

What does a PO/BA do here?

In traditional products, a business analyst works with features, user scenarios, and stable acceptance criteria. In synthetic data projects, the work shifts toward hypotheses, visual parameters, scene variability, labeling logic, and recognition-quality analytics.

In classic software development, a business analyst defines the scope of functionality: what the system should do, which screens it should show, and how it should behave in response to user actions. In projects involving synthetic data, this logic changes because the actual object of work is no longer a business function or user actions, but rather visual content intended to teach the model to better recognize specific object classes or scenarios. It is actually closer to game development, where the focus is less on requirements and more on scenarios.

For example, if the goal is to improve recognition of camouflaged tanks in order to increase targeting accuracy, the main difficulty is not in “programming a function,” but in defining:

• which visual scenarios need to be modeled?

• which environmental variations are missing?

• how the object appears under different conditions;

• how angles, lighting, occlusion, background, and visibility change;

• how these scenes should be labeled;

• and most importantly, whether the generated content actually improves recognition quality.

In such projects, scope is not just a list of requirements. It is a visual framework. In practice, the team goes through several sequential layers of work:

• creating the base scene and the main object;

• building the environment in which these objects exist;

• modeling both typical and atypical scenes;

• adding variability;

• defining labeling rules [the identifiers of what the AI needs to learn];

• validating on the model;

• correcting the scope based on recognition results.

So the scope is never final at the start. It evolves together with the testing results. At first, the team may build basic scenes: landscape, common movement paths, standard camera angles, basic weather or lighting conditions. Then a new level of complexity emerges: partial occlusion, varied background types, scale changes, challenging lighting, night scenes, moving objects, loss of clear contours, fine details, and visual noise.

That is why, in these projects, a business analyst works not only with the question of “what needs to be built,” but also with the frame of “what kind of visual reality do we need to create so the model can learn to see what it currently fails to see.” In my own projects, this was achieved largely through trial and error.

Why do User Stories and Acceptance Criteria become more difficult?

Classic user stories only work partially in these projects. They can still be written, but they become less functional and much more descriptive.

For example, a user story might sound like this: the team wants a set of scenes in which the target object moves across different types of terrain, is partially occluded by natural elements, is observed from a top-down angle, and appears in several visual variations.

Formally, this is already a user story. But it does not carry the same level of clarity we usually expect in classical business analysis. The reason is that the main value is not the scene itself but its impact on the model's behavior.

Acceptance criteria also stop being purely binary. They cannot be reduced to a simple “works / does not work” check. They become hybrid and include several layers at once:

• visual level — the scene matches the required environment, angle, lighting conditions, and level of variability;

• structural level — all required objects are present, have the necessary parameters, and are organized correctly;

• labeling level — the objects are labeled according to the rules, and annotations are consistent;

• analytical level — the use of the new synthetic dataset produces improvement, or at least a measurable impact, on model metrics in the target scenario;

• research level — the team receives enough signals to understand what needs to be changed next.

In a nutshell, these projects' acceptance criteria are not only requirements for content but also for the usefulness of that content for training.

Validation as the central part of the work

One of the main challenges in such projects is that creating the dataset is not the final point. In reality, it is only the beginning of the validation cycle.

The team generates the first dataset and passes it to the Data Science team. After that, the model is trained or fine-tuned on the new data. The results are then analyzed using quality metrics: precision, recall, false positives, false negatives, confusion matrices, class-level behavior, and stability of recognition under a variety of conditions.

And this is exactly where the role of the PO/BA becomes especially interesting. It is no longer enough to simply “collect requirements.” You need to understand why:

• where the model confuses classes;

• which scenes turned out to be too idealized;

• which visual variations are missing;

• whether the issue is in the scene itself, class ratios, labeling, or dataset imbalance;

• whether more realistic defects, more noise, more complex backgrounds, or different lighting conditions need to be added.

For example, a model may work well on large objects but perform much worse on small details. Or it may confidently recognize an object in daylight but lose quality at dusk, at night, or in low-contrast scenes. In such a case, the business analyst not only documents the problem but also helps transform it into a new work package: new scenes, new variations, new generation rules, and new evaluation criteria.

The balance between synthetic and real data

Another difficulty is that synthetic data rarely works best in isolation from real data. In practice, the greatest value usually comes from mixing synthetic and real data. Real data teaches the model “how things actually look,” while synthetic data teaches it “what to look for.” But this immediately creates another layer of thinking.

You need to understand:

• what proportion of synthetic and real data is optimal?

• how the model behaves under different mixes;

• whether the model starts to over-focus on the more frequent class;

• whether rare or defective states become undervalued;

• whether the synthetic content becomes too “clean” compared to reality.

If intact objects dominate the dataset, the model may start treating defective states as noise. If, on the contrary, the emphasis shifts too strongly toward defects, the model may lose quality in normal states. If synthetic scenes are too perfect, the model may learn very well on them but fail to transfer that knowledge to real-world conditions.

That is why a significant part of the work in these projects is investigation: finding the right proportion, the right set of scenes, the right level of realism, the right amount of augmentation, and the right target classes. Here, a PO has to learn to think a bit like AI. The work becomes about designing experimental logic, tracking hypotheses, structuring conclusions, and translating Data Science results into understandable business decisions.

Communication between stakeholders is a separate challenge

I constantly communicate with:

• business stakeholders, who formulate the problem but often do not understand the context or assume AI is all-powerful;

• domain experts, who explain what real objects, defects, or scenarios actually look like;

• 3D artists, who build the visual scenes;

• the ML/Data Science team, which trains the models and analyzes the results;

• annotation and labeling specialists;

• DevOps engineers, who ensure the storage and transfer of gigabytes to terabytes of data.

The challenge of organizing video creation

You also need to structure and slice the requirements so that several 3D artists can work on one scene in parallel. One person may work on bushes, another on buildings, and another on the tank's camouflage. In reality, all the stories often look almost the same textually, with the main difference being the example images inside them. On top of that, it is important to consider the reusability of objects across projects, because that can significantly speed up development.

Why Kanban often works better than classical Scrum

Planning in such projects has its own specifics as well. While some blocks of work can be planned in advance, overall, the project rarely moves in a linear way. A lot depends on the results of intermediate validation: what exactly the model failed to recognize, where false positives appeared, where recall dropped, and which scenes did not produce the expected effect.

Because of this, the backlog often looks less like a set of stable features and more like a living list of hypotheses, refinements, improvements, and research tasks. That is why a Kanban approach often feels more natural than a rigid sprint rhythm. The team is constantly reacting to new insights, adding new elements, changing priorities, and reassessing the usefulness of individual scenes or datasets.

In such an environment, the PO/BA is responsible not so much for “closing features” as for managing the logic of the workflow: what needs to be tested next, where the biggest risk is, which experiment has the highest value, and how to keep the link between the business goal and the many technical and visual iterations.

Conclusion

Projects involving SDGs for AI model training fundamentally change the classical understanding of the Product Owner/Business Analyst role. It is no longer enough to simply describe requirements, agree on user stories, and pass them into development. You have to work with a complex system of interdependencies between business goals, visual content, annotation rules, training results, and continuous research-driven validation.

In these projects, BA/PO roles become more than process ownership. They become moderators of meaning. They help transform a vague request, such as “the model should see complex objects better,” into a manageable process: from defining scenes and variability to analyzing metrics, reformulating scope, and making decisions based on results.

Art of Business Analysis training schedule 

News and articles on business analysis: 

• Telegram 

• Linkedin