1.2 Logging with Browser library
1.2.1 Python logging in log.html
Robot Framework log.html file contains mostly logging from the Python side of the library. Some keywords may
also log a limited amount of events from the NodeJS side, usually just the keyword status. However, the status
does not explain what happens in the NodeJS server or which Playwright API calls were made. If a Browser library
keyword fails, the info-level logging shows the error from the Playwright API call.
Start the test app in a separate shell with command: node test_app/server/server.js
Then navigate with your favorite browser to http://localhost:7272/prefilled_email_form.html
and see that page opens successfully.
As a first task, write a test that opens the page at http://localhost:7272/prefilled_email_form.html
and tries to click something that does not exist. Run the test with the default settings of the robot command.
When running this example:
*** Settings ***
Library Browser timeout=2s
Suite Setup New Browser headless=False
*** Test Cases ***
Example Logging On Error
New Page http://localhost:7272/prefilled_email_form.html
Click id=foobar
with command:
robot --outputdir output examples/1/1.2/example_logging_on_error.robot
But when using debug level:
robot --loglevel debug --outputdir output examples/1/1.2/example_logging_on_error.robot
It contains a more detailed error message from the NodeJS server side.
1.2.2 Node side logging
${OUTPUT_DIR}/playwright-log.txt is always created. By default, it contains logging from the NodeJS side of the
Browser library server. Setting the log level on the Robot Framework side does not affect what is written to
playwright-log.txt. More details can be gathered by enabling Playwright logging. This is done in the library import
with the setting: enable_playwright_debug=True. Please note that enabling Playwright logging logs everything
as plain text, including secrets.
From the previous example, open ${OUTPUT_DIR}/playwright-log.txt and look at it.
Edit the previous test and enable Playwright debug logs with the enable_playwright_debug
argument. Also enhance the test so that it does not fail and performs more actions on the page.
Then execute the test, open ${OUTPUT_DIR}/playwright-log.txt and look at it.
Please remember that each test execution will overwrite the playwright-log.txt file.
*** Settings ***
Library Browser enable_playwright_debug=True timeout=2s
Suite Setup New Browser headless=False
*** Variables ***
${secret} this is secret in log.html but not in other logging
*** Test Cases ***
Playwrifght debug logs
New Page http://localhost:7272/prefilled_email_form.html
Type Text [name=comment] this is not secret
Type Secret [name=comment] $secret
with command:
robot --outputdir output examples/1/1.2/example_pw_logging_enabled.robot
1.2.3 Playwright trace
Playwright provides tracing. Tracing can be enabled
with the New Context tracing=filename.zip keyword. When
pages are opened under the same
context, then each Playwright API call in the page is recorded to the trace file. Enhance the previous test and
make it open a second page. On the second page, add more actions. At the start of the test, use the New Context
keyword and enable tracing to create a trace file.
This is an example:
*** Settings ***
Library Browser enable_playwright_debug=True timeout=2s
Suite Setup New Browser headless=False
*** Variables ***
${secret} this is secret in log.html but not in other logging
*** Test Cases ***
Playwrifght debug logs
New Context tracing=${{$LOGLEVEL == 'TRACE'}}
New Page http://localhost:7272/prefilled_email_form.html
Type Text [name=comment] this is not secret
Type Secret [name=comment] $secret
Take Screenshot
New Page http://localhost:7272/prefilled_email_form.html
Type Text [name=email] This first text
Type Text [name=email] This second text
Take Screenshot
Run it with command:
robot --outputdir output examples/1/1.2/example_pw_trace.robot
Then the trace.zip file from the output directory can be opened in two ways:
- Playwright provides online service: https://trace.playwright.dev/
- Open trace file locally with command:
rfbrowser show-trace output/trace.zip
Investigate the different tabs and features in the trace viewer.
1.2.4 Coverage
Browser library provides a code coverage feature. Data for coverage is collected by Playwright. The report is created by monocart-coverage-reports Coverage must be enabled for each page separately.
Write a new test to collect coverage from google.com using the Start Coverage and Stop Coverage keywords.
*** Settings ***
Library Browser enable_playwright_debug=True
*** Test Cases ***
Playwright coverage logs page 1
New Page https://www.google.com/
Start Coverage
Take Screenshot
Stop Coverage
Run test with command:
robot --outputdir output examples/1/1.2/example_coverage.robot
Open the log.html file and see Stop Coverage keywords.
Enhance the previous test to open two pages and collect coverage from each of them. Also remember to enable raw data collection.
*** Settings ***
Library Browser enable_playwright_debug=True
*** Test Cases ***
Playwright coverage logs page 1
New Page https://www.google.com/
Start Coverage
Take Screenshot
Stop Coverage
Playwright coverage logs page 2
New Page https://www.google.com/
Start Coverage
Take Screenshot
Stop Coverage
Run test with command:
robot --outputdir output examples/1/1.2/example_coverage.robot
Open the log.html file and see Stop Coverage keywords from both pages.
Combining coverage reports can be done by rfbrowser entry command:
rfbrowser coverage output/browser/coverage/ output/report
Investigate the combined coverage report.
Report data is collected by Playwright and the report is created by Monocart Coverage Reports. Monocart offers plenty of options to tune the report for your needs. For example, there are many different kinds of filtering options. Browser library supports Monocart config file