As AI models become more complex, the need for large, high-quality annotated datasets has only grown. Many companies have relied on Scale AI to meet this demand because of its unique combination of services and a robust platform. This setup made it easy to manage data, track progress, perform QA, and integrate annotations into a larger data pipeline.
Most service providers lack this combination – they do not provide you with a way to engage with the entire data annotation pipeline. This not only increases the time you must spend overseeing the project, it can also lead to ‘labeling debt’ – you realize too late that your labeling guidelines need an update, since the annotation process is not transparent. This labeling debt can result in labeled data that is out-of-date with current requirements and necessitate re-labeling, causing delays and an increase in total cost of the project.
Why shift from Scale ?
However, recent changes at Scale AI are forcing organizations to rethink their options:
- Relationship with Meta: Meta has acquired a huge stake in Scale, and the CEO - Alex Wang has moved there as well. This raises potential conflict of interest concerns for some clients.
- Product shift: Scale has discontinued its rapid annotation service to focus more on GenAI products. Rapid was a popular choice for many teams since you could simply upload your data, and track its progress on the platform, without needing back and forth with their team. For customers who need large volumes of labeled data, this means reduced support.
However, migrating from one provider to another is not always easy. We have prepared a fun little evaluator's hand-book to assist in this decision. If you would like to read it, let us know about your requirements, and we will send it to you.
That being said, when evaluating alternatives to Scale or any other provider, it’s important to consider three key aspects:
Platform access and features
While you are looking for Annotation services providers, its important that they provide a tool to interact with the data annotation process. In particular, make sure the tool supports:
- Data management - So you can upload your data, and download the completed annotations.
- Project monitoring - So you can track progress, view reports and check the annotation workflow.
- Quality checking - When interacting with our customers, the biggest concern they often raise is the time required to perform Quality Check on the annotated data. To reduce this time, it is imperative that there is some way to do QC on the tool itself. Checking the quality offline using scripts simply does not scale.
Workforce model
All annotation teams are not made equal. Whether the company uses its own workforce, crowdsourcing, or external vendors, can determine the quality and type of annotations you get.
Pricing models
Vendors can offer different pricing models. By far the most common are a per hour model, where vendors charge based on the time spent annotating the data. The other model is to price the annotation based on each asset annotated - this could be per image, per annotation or a combination of both. In addition, vendors also differ in whether they require any minimum time or budget commitments to engage in an annotation project. Its important to understand these before you go too deep into your engagement with a vendor.
Below, we’ll look at a few leading providers — CloudFactory, Sama, Hive, Telus, and our own platform, Mindkosh — to help you understand their strengths and differences. While each option has its unique value, we’ll save a deeper dive into Mindkosh for the end of the article. Note that some of the information presented below might not be publicly disclosed by the companies, and has instead been collected from user reviews and other independent reports.
CloudFactory
Overview
CloudFactory is a well-known name in the managed workforce space. Founded in Nepal in 2010, the company has since grown into a global provider of data labeling and workforce management services. Their workforce is largely based in Nepal and Kenya, though they serve clients worldwide.
Platform and Features
CloudFactory combines its workforce with a platform that allows basic project oversight:
- Data management: Clients can upload data, download annotations, and connect their own cloud storage using APIs or webhooks. Data does not need to sit on CloudFactory’s servers.
- Progress tracking: The platform includes workforce management tools, project tracking, and analytics.
- QC: Quality oversight is available but fairly basic compared to more specialized annotation platforms.
Workforce Model
They employ a mix of full-time, part-time, and freelance workers. This makes them flexible, but it also means quality may depend on the specific team assigned.
Pricing
CloudFactory charges primarily by the hour. While this model works for certain use cases, it can reduce transparency. Clients sometimes find that per-hour billing encourages longer project timelines, rather than efficiency.
Sama (formerly Samasource)
Overview
Sama is one of the oldest players in the data labeling industry. Founded in 2008, the company positioned itself as a social impact organization by employing workers in developing countries, particularly in Kenya and Uganda. Their mission has been to lift people out of poverty by providing digital jobs.
Platform and Features
Sama offers a platform called SamaHub, which includes:
- Data management: Uploading, annotating, and exporting data.
- Progress tracking: Clients can monitor tasks and compare annotations.
- QC: Built-in tools provide slightly more annotation visibility than CloudFactory.
- API: Available for integration into existing data pipelines.
Workforce Model
Sama employs full-time workers in controlled office environments, unlike CloudFactory, which sometimes uses freelancers. This provides more stability in workforce quality.
Remarks
Despite its social impact mission, Sama has faced criticism in recent years regarding the treatment of workers, particularly on content moderation projects for Meta. Clients should weigh these concerns when choosing Sama.
Hive AI
Overview
Hive takes a very different approach compared to CloudFactory and Sama. Instead of relying on a managed workforce, Hive crowdsources data labeling through its mobile app. The company was founded in 2013 and initially focused on creating a platform for crowdsourced micro-tasks. Over time, it has pivoted towards AI-powered content moderation and automation tools.
Platform and Features:
Hive does not offer a client-facing web platform. Instead, clients interact with Hive’s services through APIs.
- Data management: Upload data and retrieve labeled outputs via API.
- Progress tracking: Limited visibility, since there’s no dashboard or annotation management interface.
- QC: Quality depends heavily on their internal workflows for managing crowd workers.
Workforce Model
Hive relies on a large pool of gig workers labeling data via its app. While this can scale quickly, it comes with variability in worker quality and less predictability.
Remarks
Hive has increasingly shifted its focus away from managed annotation towards automated content moderation and GenAI products. If your needs are annotation-heavy, be mindful of this shift and focus.
Telus International
Overview
Telus International is a global BPO (business process outsourcing) company with a significant focus on digital solutions, including data annotation. As a large enterprise vendor, Telus caters primarily to organizations that need long-term, large-scale engagements.
Platform and Features
Telus provides an in-house annotation tool as well as support for third-party annotation tools. Their offering includes:
- Data management: Uploading, browsing, and exporting data.
- Progress tracking: Analytics, dashboards, and project progress reports.
- QC: Built-in features for monitoring annotation quality.
Workforce Model
Telus uses an in-house workforce and can even deploy teams on-site if needed. This makes them a good choice for enterprise clients requiring high levels of control.
Pricing
Their contracts are usually asset-based and expensive. Telus generally requires annual or multi-year commitments, which may be less suitable for smaller organizations or projects with fluctuating workloads.
Mindkosh (Our Solution)
Overview
Mindkosh combines a powerful annotation platform with an experienced workforce. Unlike vendors that separate services from tools, we offer both under one roof—similar to how Scale worked at its best.
Platform and Features
- Data management: Upload data via Python SDK, direct cloud storage connections, or manual sharing. Export annotations in multiple formats with full access control.
- Progress tracking: Monitor live project progress, download reports and dive into detailed analytics, including labeling time spent by each user over time and object count.
- Quality Checking: Perform random checks, use a honeypot system, or apply inter-annotator agreement — all within the platform.
Specialized Support for AV (Autonomous Vehicles)
- Lidar and sensor fusion annotation.
- Support for context images alongside RGB images.
- Native support for single channel images like thermal and depth images, with colormap visualization and multi-image merging.
Workforce Model
We use our collaborations with a variety of workforce providers across India. This allows us to be flexible with the size of the team while also being able to provider customers with a single annotation team that delivers consistent, high quality annotations. All annotations come with SLA-backed quality assurance. Importantly, there are no minimum commitments, making it suitable for both small and large teams.
Pricing
Mindkosh offers a few options:
- Per asset pricing - This could be per data point, like an image (or point cloud frame), or per annotation (like a bounding box, or cuboid)
- Per hour pricing - In some cases, its difficult to price a project based on the number of assets labeled. For such cases, we offer a per hour pricing as well.
- Bulk pricing - For large projects, we also offer a flat price for the entire project.
Importantly, we have no contract minimums for budget or duration of the project. You can approximate how much your project will cost with Mindkosh by using our cost estimator tool here.
Migration from Scale
If you already have annotations from Scale (or another vendor), you can export them and we will help migrate your raw data and annotations into the Mindkosh platform without disruption.
Parting thoughts
Scale AI’s pivot has left many companies searching for new partners for annotation services. The right choice depends on your priorities:
- CloudFactory: Flexible workforce, but less transparency with per-hour billing.
- Sama: Mission-driven with stable full-time workers, though recent controversies may concern some clients.
- Hive: Crowdsourced and API-driven, but shifting away from annotation focus.
- Telus: Enterprise-grade with high cost and long-term contracts.
- Mindkosh: Balanced option with strong platform features, AV-specific support, flexible contracts, and in-house workforce.
If your goal is to replicate the best parts of Scale—strong platform plus high-quality services — Mindkosh offers the closest match, with additional flexibility for modern AI pipelines.
