Test in the Time of Change
Let me start by quoting the well-known adage by the ancient Greek philosopher Heraclitus: "Panta rhei." Roughly translated? "The only constant in life is change." And oh my, have I experienced change here at Dashlane since joining about 2 years ago!
From opening a 3rd office in Lisbon (plus expanding our Paris and New York offices), hitting an inflection point, watching a number of colleagues join (and some leave -- sad) and adapting to work remotely, this has been quite a ride.
In this article I'd like to tell you how some major changes affect our work in the QA team, and how we are adapting our testing and process to ensure quality remains on the top of our minds. Do read on!
Exponential Team Growth & Reorganization
Back when I joined in 2018, Dashlane was a team of about 125 folks, split across 2 offices. Little did I know that would be the year of our 3rd office opening, in Lisbon.
Then in July 2018 we released Dashlane 6, which is our latest major version update since we announced Dashlane 5, back in November 2017.
Ever since we've been focusing on more targetted releases rather than another "big bang", although we still have achieved a number of important milestones and deadlines. This was an opportunity to improve our release and continuous delivery process; besides making it lighter, by automating or eliminating some dependencies, we migrated our CI/CD to gitlab and changed the version numbering so that we could better adapt to weekly releases.
Fuelled by these changes and the inflection point we've hit in 2019, we have seen an accelerated growth in the team globally. While it is exciting to have more resources to do more and better things, this doesn't come without its challenges, especially if you consider that we are now over 320 Dashlaners worldwide. (Yes, we grew 2.5x in about 2 years).
Let's get back to the launch of Dashlane 6: the QA team was all based in Paris, and consisted of a small team of internal QA, supported by a small external QA team. We were mainly testing on 7 apps, for which 7 feature teams were actively contributing. (No direct relation there, as they were organized by business areas). By the end of 2018 we decided to change our team organization and evolved into mission teams.
Fast forward to 2020, and we now have 13 scrum teams, besides the platform squads. The QA team is now present in our 3 offices, and has doubled in size.
Key takeaway: Growing the number of people, teams and projects increase the workload and complexity of tasks for QAs.
- Be smart and account for the time invested in the hiring process:
- Be thorough in the pre-screening process, so that the right candidates are identified before getting to the interview stage.
- Each candidate will go through about three 45-minute technical interviews, conducted by 2-3 QAs, who also need time to prepare and debrief it.
- Having a pre-set list of exercises and questions to ask candidates saves time, but leave enough room for a personalized interview.
- Have a solid onboarding and training plan for new hires: this will help them get up to speed quickly and feel confident to take ownership and suggest improvements.
- Evolve the team organization, allow for growth and avoid knowledge silos:
- As the team grows, ensure the communication still flows consistently between projects and offices.
- Promote a continuous learning culture, with adequate training, certification and participation in events.
- Keep the team connected and flexible, so that there's always someone who can serve as a backup.
Rebranding & Super Bowl LIV
We didn't know yet to what extent, but by January we were sure 2020 would be an unforgettable year for Dashlane.
This was not the first time we've made adjustments to our image, but after celebrating one decade of existence, we decided to get a complete facelift and embarked on a massive rebranding campaign, changing our visual identity across the board. We bid farewell to the iconic impala (in case you were wondering – it was neither a deer, nor a rabbit, nor an alpaca. Seriously, an alpaca?!), introduced the "Dashlane D," changed the color scheme, added motion design, created in-app messaging and… well, let's just say we kept our name.
Now remember that we have 7 platforms / apps to maintain, and we've fully redesigned our website. That alone is a handful to test. But the impacts of a massive rebrand stretch further on, to our B2B customers, CRM & emails, user support team... and everywhere there was yet an elusive impala to be found. Actually it became a kind of a private joke / internal game to "spot the impala" for the months following the change.
This was a huge effort done by everyone in Dashlane in a relatively short period of time, between receiving marketing's green light to actually shipping this D-boat into the online and offline world (we got new super cool swag, yay!). Why couldn't we buy more time and instead had a hard deadline? Because we wanted to get the word out by February 2nd, during Super Bowl LIV.
Image credit: Catherine Gibson
We had the challenge of supporting millions of page visits in a couple of minutes as per the forecast. And we had to do multiple rounds of load testing to ensure that we can support the load. It would have been humanely impossible for the QA team to have done all this work alone, so we more than gladly share the credit for this achievement with the global team.
Key takeaway: Some cosmetic adjustments happen periodically, but not a full-blown rebranding or ad campaign. This typically requires an eye (or many!) for detail and full collaboration across teams (technical & non-technical).
- Test all the UI changes (logos, color schemes, motion) everywhere: you will be surprised with the potential for disaster that goes in changing even the smallest design aspects and how much brand and design legacy and complexity is accumulated over the years.
- Test integration with different platforms, screen sizes and entry points: the often-heard "works on my machine" will become your most recurring nightmare; it might break when you try it on a funny-sized tablet, or when you switch browsers, or when you are a user whose premium plan just expired, or...
- Do some serious performance and stress testing: as we saw our Super Bowl ad airing at different times and locations, our servers were spiking with requests from all around. our server team had prepared for weeks in advance and adjusted for such an increase in usage; not only that gave us peace of mind, but allowed all our systems to run smoothly.
- Coordinate the release of the new brand to existing users: you don't want part of your users to be caught by surprise with a new design (some did think was some hacker's work), while others are stuck to an "old-fashioned" version of your self; this is a challenge, namely aligning on a time slot and having to go through the app stores' approvals.
The Year of Working Dangerously Remotely
Spoofing attacks are something we know about, but here I'm just spoofing the Australian novel/film bearing a similar name. That said, the COVID-19 pandemic did appear to make online life more dangerous, and the all-remote work we've been experiencing for the last 6 months has brought some challenges to the way I was used to operate.
As mentioned before, our QA team is split across our 3 offices, plus our external team. That makes for 4 different time zones, a situation that comes with its own set of pros & cons and to which we are accustomed. What I wasn't used to having was all this physical distance within the same office and the development teams. Please allow me to adapt the Common Era (CE) notation system (now rebranded as the COVID Era) to describe some of the aspects that changed for me as a QA team member:
Key takeaway: Going fully remote has the potential to disrupt a team's working habits. Not everyone is fit to work from home all the time, and this particular period of confinement can hinder personal performance.
- Over-communicate: keep everyone in the loop and document decisions + knowledge as much as possible. prefer live communication (video or chat).
- Be proactive and get involved, especially for production or otherwise critical issues; it's key to stay up to date on what's happening to the product.
- Pay extra attention to your teammates, as you're missing a lot of info which you would get from non-verbal comm and the occasional coffee-machine chat.
- Take regular breaks and time off: it's easy to fall prey of zoom fatigue and work well past your schedule, as office and home blend into one same space.
Embracing Change
Let me go back to the first idea, and reinforce that the question is not "if" changes will come, rather than "when" that will happen.
Your job is to anticipate or, should that not be possible, be prepared to adapt quickly to the new circumstances. In our case that meant:
- Going from "big bang" to more targeted releases, while improving our release process.
- Growing and reorganizing the team, putting the right amount of effort in hiring and training.
- Teaming up for a company-wide rebranding, focusing on UI, performance and integration testing.
- Quickly adapting to remote work, being attentive, proactive and over communicating with the team.
Sure, going through a time of big changes is no walk in the park: things will stop working, your patience will be tested, stress levels will go up and your energy will go down. But embracing change will make your team and culture stronger, if you approach it as a chance to grow and evolve.
Sign up to receive news and updates about Dashlane
