Testing and debugging

The Momentics IDE for BlackBerry provides a source-level debugger that's integrated with the other workbench tools.

The IDE debugger uses the GNU Debugger (GDB) as the underlying debug tool. The IDE debugger translates each action in the UI into a sequence of GDB commands, and then processes the output from GDB to show the current state of the application being debugged. You can see and control the GDB commands by using the GDB consoles that are available in the Debug perspective in the IDE. For more information, see Using GDB.

You can also perform other debugging tasks, such as:

Debugging is supported for both C++ and QML/JavaScript. For more information on debugging C++ projects, see Use the debugger. You can find more information about debugging basics, such as:

Keep reading to learn about the following topics:

If you are writing apps for the work space (that is, developing enterprise apps for an organization), see Testing and debugging apps in the work space.

Debugging a QML application

Before you can start the debugger, you must build binaries using the correct build configuration (either Device-Debug or Simulator-Debug) and you must create a debug launch configuration for the target you want to debug the application on.

By default, the JavaScript debugger is disabled. To debug QML/JavaScript code, you must enable JavaScript debugging in the Momentics IDE:

  1. Click Window > Preferences.
  2. Expand > BlackBerry, and click QML Debugging and Profiling.
  3. Select Enable JavaScript Debugging.
  4. Click Apply.
  5. Click OK.

By default, the Wait for connection check box is selected. If your app is deployed in debug mode with JavaScript debugging enabled and the Wait for connection check box selected, your app won't start on the device if you try to open it outside of the IDE. This behavior is expected because the JavaScript debugger waits for a connection to your app. If you try to open your app outside of the IDE, your app won't start because there is no connection with the JavaScript debugger.

If the Wait for connection check box is cleared, the JavaScript debugger doesn't wait for the initial connection and your app starts on the device when it's launched in debug mode. When the Wait for connection check box is cleared, a breakpoint that is set in initialization code will never be hit. For example, if you set breakpoints in initialization code that runs when the creationCompleted() signal is emitted, these breakpoints won't be hit.

Screen showing the QML Debugging and Profiling preferences in the Momentics IDE.

When you add a new breakpoint in QML/JavaScript code, you are prompted to enable JavaScript debugging.

To start the debugger:

  1. Make sure that your project is showing in the Active Project window or select it from the drop-down list.
  2. From the Action drop-down list, select Debug.Debugging a QML application
  3. Click Screen showing the debugging toolbar in the Momentics IDE..

You can also right-click your project and select Debug As > BlackBerry C/C++ Application.

The Momentics IDE switches to the Debug perspective, installs the app on your simulator or device, and starts your app under the control of the debugger. In the Debug view, you can see an overview of your process, including the call stack. You can control the debugger by using the buttons on the main toolbar of the Debug view. In an application that contains both C++ and QML, the debugger is able to pass back and forth between the two types of source code.

Screen showing Debug view in the Momentics IDE.

By default, the debugger stops on the first line of your application. To change this behavior:

  1. Right-click your project and select Debug as > Debug Configurations.
  2. Select the debug configuration you want to change.
  3. Click the Debugger tab.
  4. Clear the Stop on startup at check box.
Screen showing the Debugger options that can be used to change the behavior of the debugger when it starts.

Debugging with the console and the editor

There are other ways that you can debug your app in addition to using the debugger. These methods involve using the console and editor to perform the following tasks:

Use console.log() to output to the console

In QML/JavaScript, you can use the console.log() function to output text and variables to the console.

For example, in the CircularSlider tutorial, the app captures the valueChanged() signal and outputs the slider value to the console when the value changes:

CircularSlider {
    layoutProperties: DockLayoutProperties {
        horizontalAlignment: HorizontalAlignment.Center
        verticalAlignment: VerticalAlignment.Center
    }
    onValueChanged: {
        console.log("Slider value: " + value);
    }
}

Use qDebug() to output to the console

In C++, a common way to output text and variables to the console is by using qDebug(). Here's an example of how QCompass readings are displayed:

 QCompassReading *reading = m_CompassSensor->reading(); 

qreal azimuth = reading->azimuth();
qDebug() << "The azimuth is " << azimuth << " degrees."; 

Use fprintf() to output to the console

You can use the fprintf() function with stdout or stderr as the output stream to send output directly to the console. Or, you can use a function similar to the following:

void myMessageOutput(QtMsgType type, const char* msg){
    fprintf(stdout, "%s\n", msg);                
    fflush(stdout); 
} 

Then, you can register this handler function with qDebug by calling the qInstallMsgHandler() function in your main function after the default Application is created, similar to the following:

int main(int argc, char **argv)
{
    Application app(argc, argv);
    qInstallMsgHandler(myMessageOutput);

    // ...

}

Now, qDebug() calls are logged to the console.

Console logging and qDebug() output debug messages to the console only when the myMessageOutput() function is declared and registered. You should remove this code when the application is ready for release.

Use Qt debugging macros and functions

The QtGlobal header file contains some useful debugging macros and functions in addition to qDebug():

  • Q_ASSERT() is useful for testing functions that return a Boolean value.
  • qWarning() displays a warning message in the console.
  • qFatal() closes the app and creates a core dump that you can use for debugging.

You can use these functions and macros to test various conditions in your apps. For example, consider signal-slot connections in code samples. To ensure that your app works as you designed, you must evaluate the Boolean value that the QObject::connect() function returns. A signal-slot connection fails if the sender or receiver objects are invalid or the types of the signal arguments are not registered in Qt as meta-types. In these cases, Qt generates a warning that is posted to slog2info log files on the device.

You should use the Q_ASSERT() macro to evaluate the return value of connect(), as suggested in Using predefined signals.

This technique affects only the debug builds of your code because the release builds define QT_NO_DEBUG, which disables Q_ASSERT(). Because all apps and libraries (including Qt) are released in release mode (not debug mode), the handling of the connect() return values using Q_ASSERT() doesn't affect published apps. This technique has no code to recover from a failed connection because it assumes that there is no safe recovery.

// If any Q_ASSERT statement(s) indicate that the slot failed 
// to connect to the signal, make sure you know why this happened. 
// This is not normal, and will cause your app to stop working!
bool connectResult;
  
// Since the variable is not used in the app, this is added to avoid 
// a compiler warning.
Q_UNUSED(connectResult);
  
connectResult = QObject::connect(smokeDetector, 
                                 SIGNAL(smokeDetected()), 
                                 sprinkler,     
                                 SLOT(dispenseWater()));
                                   
// This affects only Debug builds.
Q_ASSERT(connectResult);

You can also check the return value of the connect() function using an if statement and add some code to recover from the failed connect(). You can use qWarning() to send a message to the console. This technique should not be used to recover from coding errors. If a signal-slot connection fails because of an error in code, it should be fixed before it is released.

if(!connect(...)) {
   qWarning("Recovering from the failed connect()");
   // Add your code here to make sure that your app 
   // still works correctly, even though this connection 
   // has not been established.
}

You can also set the QT_FATAL_WARNINGS environment variable in your bar-descriptor.xml file. If the QT_FATAL_WARNINGS environment variable is set to 1, your app closes after qWarning() prints the warning message. For more information, see The bar-descriptor.xml file.

If you think that recovery is impossible because a successful connection is critical to your code, you can use qFatal() to close the app immediately. You can also consider adding additional code to clean up before closing the app.

if(!connect(...)) {

   // If your app uses shared files, write details about what
   // happened into a log file and ask users to send it to you.
   writeErrorLog();   
   
   // Make sure that you save as much user data as possible.
   saveOpenUserData();
   
   // Consider notifying users about the fatal error and 
   // warning them that the app will close now.
   qFatal("Cannot recover from the failed connect()");
}

Display Qt variables in the editor

In the Momentics IDE, you can select an option to improve the display of Qt-based objects and variable values while you are debugging. This option lets you view the values of Qt objects, such as QVariant and QVariantMap, that you send to the debug console.

  1. In Windows and Linux, click Window > Preferences. In Mac OS, click Momentics > Preferences.
  2. Expand BlackBerry and click Debug.
  3. Select the Use advanced scripts to show Qt variable values check box.
  4. Click Apply and then click OK.

Unit testing with QTestLib

The QTestLib framework is a tool for unit testing Qt-based applications and libraries. You can use QTestLib to test individual functions in your app. These tests are compiled into a test binary, which is compiled against the functions that you want to test. Because the tests are compiled into a separate binary, your final product doesn't contain any traces of test functions. For more information, see the QTestLib Manual on the Qt website.

Set up QTestLib

Add CONFIG += qtestlib to your .pro file to compile QTestLib into a test binary. A number of macros exist to verify your result, including QVERIFY and QCOMPARE.

Copy cascadestester.hpp and cascadestester.cpp to the src directory of your project. These two files implement the CascadesTester class. The macro QTEST_CASCADES_MAIN implements a main function and registers the CascadesTester class.

#ifndef CASCADESTESTER_HPP
#define CASCADESTESTER_HPP

#include <bb/cascades/Application>
#include <bb/cascades/Page>

#include <QtCore/QObject>
#include <QtTest/QtTest>

/**
 * @short A helper class to execute QTestLib based unit tests in Cascades
 *
 * This class executes a QTestLib based unit test inside the main event loop
 * of Cascades, so that Cascades controls can be tested.
 */
class CascadesTester : public QObject
{
    Q_OBJECT

    public:
        /**
         * Creates a new cascades tester object.
         *
         * @param object The unit test object.
         * @param argc The count of the command line arguments.
         * @param argv The command line arguments.
         */
        explicit CascadesTester(QObject *object, int argc = 0, char **argv = 0);

    public Q_SLOTS:
        /**
         * Starts the execution of the unit test.
         */
        void startTest();

    private:
        QObject *m_object;
        int m_argc;
        char **m_argv;
};

#define QTEST_CASCADES_MAIN(TestObject) \
int main(int argc, char **argv) \
{ \
    bb::cascades::Application app(argc, argv); \
    TestObject tc; \
    CascadesTester tester(&tc, argc, argv); \
    bb::cascades::Application::instance()->setScene(new bb::cascades::Page()); \
    QMetaObject::invokeMethod(&tester, "startTest", Qt::QueuedConnection); \
    return bb::cascades::Application::exec(); \
}

#endif
#include "cascadestester.hpp"

#include <bb/cascades/Application>

#include <QtTest/QTest>

CascadesTester::CascadesTester(QObject *object, int argc, char **argv)
    : m_object(object), m_argc(argc), m_argv(argv)
{

}

void CascadesTester::startTest()
{
    // start the unit test
    const int result = QTest::qExec(m_object, 1, m_argv);

    // terminate application
    bb::cascades::Application::exit(result);
}

Create tests

You can define a class for your test by subclassing QObject and adding private slots to it. Each private slot is a test function in your test. You can use QTest::qExec() to run all test functions in your class. Here's an example of how to create a class for your tests:

    class TestMyObject: public QObject
    {
         ....
         private slots:
             void test1();
             void test2();
         private:
             MyObject* p;
      }

Then you can create tests. For example:

 void TestMyObject::test1()
 {
     p->doSomething();
     Q_VERIFY(someConditionMet());
  }

In your main.cpp file, you can add the following line:

TestMyObject test1; 
QTest::qExec(&test1, argc, argv)

You can use Qt Creator to run a suite of tests. Your code just needs the following lines:

#include "AutoTest.h"

// .... 

DECLARE_TEST(TestMyObject);

AutoTest::run(argc, argv)
.

Data-driven unit testing

When you are testing a certain function, it might be useful to test it with different arguments. You can do this testing with calls to QCOMPARE. For an example, see the FilterProxyDataModelTest in GitHub.

Or you can use QTestLib and data-driven testing. For example, one function is the actual test case and another function, with the suffix _data, creates the data set. The test function is invoked for each row in the data set.

In the _data function, you set up your data using QTest::addColumn() and create test data using QTest::newRow(). For example:

QTest::addColumn<QString>( "input" );
QTest::addColumn<QString>( "result" );

QTest::newRow( "all lower" ) << "hello" << "hello";
QTest::newRow( "mixed" ) << "HellO" << "hello";

In the test case, you get the data using the QFETCH macro:

QFETCH( QString, input );
QFETCH( QString, result );
QCOMPARE( input.toLower(), result );

You can use your own data types if they have been registered using Q_DECLARE_METATYPE.

You can set up the macro QEXPECT_FAIL to expect failure from a given piece of data in the set. In the following code sample, the first parameter is the name of the data value, the second parameter is a message to display, and the third parameter tells the app whether the test case should continue after an error.

QFETCH( QString, data );
QFETCH( int, result );
QEXPECT_FAIL("expected error",
             "To be fixed", 
              Abort );
QCOMPARE( data.count(), result );

The QObject emits a signal in response. The class QSignalSpy is designed to verify emitted signals. The class inherits QList< QList<QVariant> >. The outer list's items each represent a signal emission, while the inner list's items represent the signal's arguments.

Create benchmarks

Test functions can be benchmarked.

  1. Apply the QBENCHMARK macro to parts of test functions.

    There can be at most one QBENCHMARK for each test function.

  2. Run the test case in the usual way.

For example:

class MyFirstBenchmark: public QObject {
    Q_OBJECT
private slots:
    void myFirstBenchmark() {
        QString string1, string2;
        QBENCHMARK {
            string1.localeAwareCompare(string2);
        }
    }
};

Use backends

Backends measure and report the performance of benchmarks. These backends are selected using command-line options:

  • The default walltime backend measures and reports elapsed time.
  • The -tickcounter option selects the CPU tick counter backend.
  • The -eventcounter option selects the event counter backend.
  • The -callgrind option selects the Callgrind backend, which is specific to Linux.

Some backends run benchmarked code multiple times to ensure accurate timings.

For more details, see the QTestLib Manual on the Qt website.

Testing and debugging apps in the work space Since 10.2

If you develop enterprise apps, you can test your apps in the work space of a BlackBerry 10 device that is running BlackBerry 10 OS version 10.2 or later. In the development environment you use, set your launch configuration to run or debug on a device, and switch to the work space before you deploy the app.

There are a few considerations when you test apps in the work space:

  • You must be running BlackBerry 10 OS version 10.2 or later on your BlackBerry 10 device.
  • The Restrict Development Mode IT Policy rule must be set to No.
  • The Development Mode Access to Work Space IT Policy rule must be set to Allow.

    Only BlackBerry Enterprise Service 10 version 10.1.3 or later supports the Development Mode Access to Work Space IT policy.

  • If the Development Mode Access to Work Space rule is disabled, apps that you previously tested in the work space are removed.
  • You can overwrite an app that your administrator pushes, but if the Development Mode Access to Work Space rule is disabled, your app is removed and isn’t reinstalled.

For more information about developing enterprise apps, see Developing Enterprise Applications.

Last modified: 2014-01-23

comments powered by Disqus