Codeless vs. Coded Test Automation Is a Dated Debate

Published in

FAUN — Developer Community 🐾

5 min readAug 10, 2021

Codeless test automation has been a point of heated debate since the technology arrived on the scene. One only needs to do a quick Google search to find the latest “coded vs. codeless” comparison.

This is a dated debate in many regards. There’s room for both.

Codeless tools have matured significantly since the early record-and-playback solutions. It is not one or the other for most organizations.

Instead, we can accept that codeless automation is here to stay and is beneficial in different ways than coded tools, especially when applied properly. We can then look at the bigger picture and see where codeless technology best fits in our overall DevOps ecosystem.

Background: Codeless Test Automation’s Metamorphosis

Many have viewed codeless test automation with skepticism at best or disbelief at worst.

Much of this initial ambivalence can be attributed to record-and-playback solutions, or recorders. As the “codeless testing tools” of old, recorders left a lot of frustration in their wake by working for simple test scenarios by ultimately causing more problems when trying to scale.

As Angie Jones notes in her now-famous article in TechBeacon,

“As we were automating against applications that were still under development, maintenance of these automated scripts became a nightmare. Previously recorded web objects were unrecognizable by the tools as the application changed, and updating a test script often required re-recording the entire test.”

Testing teams were frustrated. The tools that were ultimately supposed to empower testers could not even handle slightly complex scenarios.

So naturally, the new age of codeless automation tools was met with some ambivalence. Often relegated to “just another record-and-playback solution,” many feared that codeless automation would present the same challenges that recorders did in the past.

For this and other reasons, many users would often turn to code-based solutions.

Some teams genuinely prefer code-based testing tools. It works for them. Especially for software engineers who like to code tests themselves, codeless testing won’t necessarily be on their radar.

It makes sense. I get it.

But for others, code-based tools just feel like a safer option since there is no fear of the unknown. They are missing out on the benefits of codeless automation because they are nervous to venture into something new.

Why Codeless Automation Is Worth It (Really)

After seeing many organizations across a wide range of industries finding success with codeless automation, I have personally seen the benefits that come with codeless testing.

Some of the major benefits of codeless automation include:

Accessibility to any team, regardless of coding skill.
Simple, visual test creation.
Reusable test steps.
Self-healing capabilities with AI and machine learning.
No framework setup.
Cross-browser executions at scale.
Ability to schedule tests within your CI/CD pipeline.
Intelligent reporting and debugging.

Many of the fears that come with record-and-playback tools are allayed by today’s codeless automation solutions. Codeless automation tools have the same ease of use as a recorder while being able to grow with your organization as you test more complex, variable, and in-depth scenarios.

In addition, codeless offers various features that are not found in code-based tools, especially when it comes to test maintenance. There is certainly no self-healing aspect of code-based tests. And as a small plug for Perfecto’s codeless automation solution, Perfecto Scriptless, it also supports more complex test elements such as Selenium code libraries, Shadow DOM, and iFrames.

So instead of asking whether to use codeless or coded solutions, it makes much more sense to think about where it is best to incorporate codeless testing into my testing activities.

The Big Picture: Coded + Codeless

One of the biggest benefits of codeless test automation is its unique ability to complement code-based testing. Instead of working against each other, codeless can make the testing process easier, more intuitive, and more strategically sound while expanding the overall test coverage.

Enterprises are dealing with a wide range of applications, some internal, some external, and are often a mix of web and mobile-based apps. Consequently, the toolchain and testing strategy cannot be one-size-fits-all.

It therefore makes sense that you will often find different testing frameworks (both open-source and commercial) within enterprises that match their teams’ existing skills, objectives, and application types.

Quality is top of mind for all enterprises. They aim for a high degree of test automation and coverage across a larger scale of devices/web browsers to ensure they reduce as many risks as possible.

A healthy mix of codeless and code-based testing solutions can help organizations achieve these quality goals at scale.

Which begs the question, how would doing both code-based and codeless testing work?

When & How to Implement Codeless Testing

There are several types of tests that would really benefit from running codeless.

The first tests to transition to codeless are manual tests that are either difficult to implement or require greater in-house skills. “Flaky” tests that continuously lead to inconsistent results (e.g., pass and fail for the same configuration or result in false negatives) are another great place to start.

Typically, a lot of these “flaky” tests are those that look at web pop-ups for unconfigured image elements or the effects of mobile behaviors, such as certain devices being blocked by certain applications. Traditionally, this includes tests that have third-party dependencies and complex prerequisites from a setup perspective. Using codeless tools that can overcome these changes automatically with AI will make these tests much more stable and scalable.

*An example of how machine learning algorithms in codeless testing tools keep tests stable and running smoothly.*

To successfully implement codeless test automation solutions into their pipeline, organizations need to keep in mind that they will have a combination of code and codeless tests that need to work together. The best approach is to exclude the tests that will operate codeless from the actual code itself, apply the codeless test, and then bring it back into the sequence.

It’s also important to note, however, that codeless automation will never completely replace manual testing. There are always going to be tests that need the “human touch,” or to be monitored much more closely.

Bottom Line

Codeless testing tools have gone through a lot over the last few years.

Yet despite the skepticism and the struggles to address the shortcomings of previous solutions, codeless tools have become an important asset that can help organizations achieve both comprehensive and strategic testing.

Whether your team wants to start with automated testing or is looking to scale, it is worth keeping codeless testing tools on your radar. They are not only beneficial on their own, but codeless automation tools also work as a great complement to code-based testing.

If this post was helpful, please click the clap 👏 button below a few times to show your support for the author 👇