![](\"https:\/\/linphone.org\/sites\/default\/files\/4_4.png\")
\nDoing all this while avoiding the risk of introducing a regression or a new defect represents major challenge.<\/p>\n
We estimate that about one third of our developers\u2019 efforts are focused solely on enhancing our non-regression procedures and maintaining our test infrastructure to ensure it is reliable and efficient.<\/p>\n
In this article, we present an overview of\u00a0our company strategy in terms of non-regression testing and quality procedures<\/strong><\/span>, which can be divided into the following categories:<\/p>\n All automated tests are conducted using\u00a0Gitlab<\/span><\/a><\/strong><\/em><\/u>\u00a0each time a developer makes a modification to the source, via a “Merge request” process, which also includes a code review by a peer developer.<\/p>\n We give a bit more detail on each category below.<\/p>\n 1. Low level, per component, automated tests<\/strong><\/u><\/p>\n Linphone SDK<\/strong><\/span>\u00a0and\u00a0Flexisip<\/strong><\/span>\u00a0are made from several software components that are assembled together. For example,\u00a0oRTP<\/strong>\u00a0is our library for implementation of the Real Time Transport protocol, belr is a small library used for parsing protocol languages, and\u00a0mediastreamer2<\/strong>\u00a0is our library for streaming audio and video over the network. These software components have their own test suites as part of their source code, which are designed and maintained by developers to assert that their basic functionality is achieved. When networking is involved, we use a local loopback network, which is simple to understand for anyone familiar with software development.<\/p>\n 2. Sophisticated automated tests<\/strong><\/u><\/p>\n Liblinphone<\/strong><\/em><\/u><\/a><\/span>\u00a0is a library whose purpose is to realise\u00a0VoIP calls, conferences and IM<\/strong>. A real-world VoIP call involves many different aspects, such as signalling protocols, media streaming, network, routers\/firewall, and servers. Our objective is to create\u00a0automated tests<\/strong>\u00a0that match the real-world usage of Liblinphone as far as possible, which is why we built not only tests on top of the Liblinphone API, but also a complete infrastructure comprising our server products, dedicated to automated testing. This test infrastructure includes:<\/p>\n The test server infrastructure is deployed based on\u00a0Ansible scripts<\/strong>.<\/p>\n We developed\u00a0specific tools together with the Liblinphone test suites<\/strong>\u00a0to assess the performance and quality of the calls\/conference\/IM flows:<\/p>\n The Liblinphone test suite<\/strong>\u00a0runs approximately\u00a01700 tests<\/strong><\/span>, which include real calls, conference and IM flows in real time, and which cover both typical end-user usage and more advanced usage such as call transfer and customer-specific SIP flows. The test results for each release (including minor ones) are published on our download page (for example,\u00a0see\u00a0one of our latest Liblinphone version 5.2 reports<\/a><\/strong><\/em><\/u><\/span>).<\/p>\n The principle of the Flexisip test suite is similar: SIP call flows generated by Liblinphone are passed to the Flexisip instance under test, which allows Flexisip to be tested in scenarios that include the entire network stack. In addition, the test suite models the use of Flexisip in a real system comprising the MariaDb and Redis databases, so that Flexisip can be tested automatically in scenarios that are very similar to real-world deployments.<\/p>\n Our testing infrastructure is constantly evolving: for example, a compilation with an\u00a0address sanitiser\u00a0<\/strong>is in the process of being added, which will mean that the test results can immediately highlight memory faults or leaks.<\/p>\n 3. UI automated tests<\/strong><\/u><\/p>\n Our UI team has developed a framework for the automation of application-level tests by simulating taps on real or emulated Android & iOS devices, and\u00a0making asserts on the rendered user interface. The\u00a0test scenarios represent typical<\/strong>\u00a0user scenarios, and involve creating or configuring a SIP account, making a call, receiving a call, sending an IM, etc.<\/p>\n 4. Manual tests<\/strong><\/u><\/p>\n Although most things can potentially be automated, at some point the cost of automation exceeds the cost of manual testing. Our app release process also includes a two-day intense testing program using multiple devices: iPhones, Android phones from various manufacturers, laptops with the Mac, Windows and Linux operating systems, and Bluetooth peripherals.<\/p>\n Conclusion<\/strong><\/u><\/p>\n The extremely large\u00a0combinatory\u00a0inherent to communications scenarios means that automated test procedures are required for all of our products. Over the years, our company has built a\u00a0complete test infrastructure<\/strong>, which enables an impressive number of test cases that reproduce real-world conditions and usage to be executed in less than half an hour. This test infrastructure is the key to ensuring strong quality assessment and quick delivery of our releases, especially in the context of our support service.<\/p>\n Don\u2019t hesitate to\u00a0contact our team<\/span><\/strong><\/u><\/a>\u00a0for more information!<\/p>\n\n
\n
\n
\nSince a call typically lasts between five and 10 seconds, tests are parallelised over multiple threads to prevent the suite from taking too much time. The full test suite can be executed in around 20 minutes, which is compatible with our strategy of agile development.<\/p>\n
![\"\"](\"https:\/\/linphone.org\/sites\/default\/files\/nous_contacter_31_x_11_cm_26_x_11_cm-2.png\")
<\/a><\/p>\n","protected":false},"author":10,"featured_media":13274,"parent":0,"template":"","meta":{"_acf_changed":false,"inline_featured_image":false},"actualites-category":[42],"class_list":["post-13272","actualites","type-actualites","status-publish","has-post-thumbnail","hentry","actualites-category-tech"],"acf":[],"yoast_head":"\n