Directory layout showcase for simple verification

Lets show a directory layout for a little imaginative rtl-project with verification testbench. Here is a bit of terminology:

DUT - design under test, the RTL project we are going to verify.

Harness - it's a module where DUT plus all verification agents are instantiated.

Verification environment(VE) - harness plus scripts required to compile harness, run simulation, run post-processing, dump and view waveforms, run one testcase in semiautomated way multiple times, etc. Writing such scripts may seem tedious but it's a must. Verification engineer developing testcases shouldn't worry about how the RTL is actually compiled/elaborated. 'make vlog sim SEED=1' knowledge is enough for testcase developer and RTL designer to run the test. Development of VE is verification integrator's job. It does require good knowledge of scripting languges.

Testcase(tc) - actually the test verification engineer is working on. Each testcase must have the same rules to compile, run. All testcases must print a unique tail with info about whether testcase passed or failed.

Teststep - testcase must be preferably splitted on a few steps. Having scores of testcases is difficult to maintain, document and track. So dealing with testcase with 3-5-7 teststeps united by common features is ok for verification engineer. On the other hand it drastically reduce number of testcases when we overview the test plan with list of testcases.

Testbench(tb) - verification environment + all testcases for DUT verification. So for verification developer testbench is a place where he/she is developing/working/running_simulation. For verification manager/team leader testbench is a final product, i.e 'RTL team is doing RTL project, verification team is doing testbench for it'.

Now get down to layout itself. The source code (directories and makefiles) is located in my repo at bitbucket and can be downloaded at tar-archive here

So the directory layout is in the picture below.


As one can see the layout reflects the DUT, harness, environment (makefiles are the scripts) and testbench.
Some important notes:

* project scripts must use relative paths. It's an inevitable requirement for multiuser, multihost environment. No 'export PROJECT_HOME=/home/myuser/projects/myproject'. Once testcases are all at the same level of hierarchy makefiles easily allow to use relative paths.

* rtl-designers, verification integrator, testcase-verificators are working in different directories on different files. Never rtl-designer should edit testbench library or tests inside testbench directory, as well as never verificators should edit files in RTL directory.

* extending idea of separation, testbench library can contain verification IPs/packages that can be considered mature and widely used in other projects. This means these packages must not be edited. All project-specific agents are going into harness directory (or harness/tb_lib in case there are a lot of such agents).

* more extending idea of separation, all testcases are located in separate directories, in case one tc-developer mistakenly commits broken testcase (non-added file or just missing semicolon), other devs won't notice it (regression will, you have it, right? :)

* having her/his own workspace (literally testcase directory) verification engineer is free to create new scripts/makefile targets, still not interfering with other scripts/make_targets from other engineers

* it becomes rather easy to create regression script to run all testcases in testbench. For example (run in testcase directory):
$ for k in ../tc_*; do echo running $k; pushd $k; make|grep -i error; popd; done

* what we do sacrifice is place(path) where we can call scripts (compilation, simulation), i.e simulation can only be started at two levels down to PROJECT_HOME

* one can say that there is extra work for harness compilation inside each testcase directory and will be right! First of all, it's extremely easy to create symlink to compiled snapshot in each testcase directory (again, it can be a make target). So it's easy to switch from one approach to another. Second, imagine I'm developing two testcases. One is finished and I want to compile harness with enabled optimization and run multiple times. The second testcase in an early stage development and I debug it with dumping waveforms. In case two testcases were located in one directory then:

1. it would require having two differently compiled snapshots (hence compilation scripts would fattened with variables)
2. running two simulations with different log-files in one directory is a nightmare.
3. developing and debugging scripts that would organize such scenario become difficult.

Tools and other resources

This post contains links for opensource projects, tools that might be interesting for functional verification, digital design modeling, etc.

resource	links	comments
Wavedrom	wavedrom editor	web-service allows to draw pretty waveforms right from browser, simple&elegant
kactus ip-xact	kactus at sourceforge	Qt-editor for ip-xact creation and MCAPI(?). The app is eye pleasing, but still lacks some features like creation memorymaps and register editing. Regretfully, guys use Qt>4.7 which may be a showstopper for enterprise solutions (scores,hundreds of working PC) where RHEL 4,5,6 are dominating OS. As of CentOS 6 yum repos contain only Qt 4.6.
sveditor	sveditor at sourceforge	eclipse plugin for systemverilog, more info about installing can be found at this blog

Books & Articles

this article will be updated on&off

General Programming

• Code Complete by Steve McConnell
• Design Patterns: Elements of Reusable Object-Oriented Software by Erich Gamma, Richard Helm, Ralph Johnson and John Vlissides (GOF)

C/C++ Programming

• Thinking in C++: Introduction to Standard C++ by Bruce Eckel (available for free download)

Python Programming

• Dive into Python by Mark Pilgrim (available for free download)

SystemVerilog & Verification

• SYSTEMVERILOG FOR VERIFICATION A Guide to Learning the Testbench Language Features by Chris Spear. The book is an excellent introduction into SV language features for verification: multithreading support, coverage & randomization features are explained with rich examples and detailed comments.
• AVM Cookbook by Mark Glasser, Adam Rose, Tom Fitzpatrick, Dave Rich, Harry Foster. Advanced Verification Methodology (AVM) is predecessor for OVM (which is in turn 'father' of UVM). AVM was developed by Mentor. Actually I wasn't able to find the book from Mentor site, probably a direct e-mail request should do the trick. The book was available for free download, a registration was required though. I truely recommend the book for beginners. It has many conceptual features for verification explained (OOP, TLM,monitors, TLM->RTL refinement, randomization). To compare, UVM cookbook doesn't explain TLM at all, it shows off examples HOW to use TLM classes, but not WHY they should be used (as of uvm1.1 uvm cookbook looks like a bunch of notes piled by different people). UPDATE: as this post points avm cookbook probably can be downloaded here after registration.

SystemVerilog & RTL Design

• SystemVerilog For Design Second Edition A Guide to Using SystemVerilog for Hardware Design and Modeling by Stuart Sutherland.

Blogs I follow

• Coolverification
• verilab

Override default uvm report server

Note: this article is just a translation from Russian one (Пример переопределения сервера ovm (uvm)).

Extending a default uvm report server can be useful in following cases:

1. send messages from uvm to your own message collector;
2. call callback at some condition, e.g. create transaction stream of errors (created with uvm_report_error) so one could browse them in waveform viewer
   
3. change message format, e.g.change modeling time format or, more useful, printing errors in red colour
   
4. replace summary report printed at the end of test run. As a rule, regression automation script greps (filters) test output and detects whether test has passed or failed. In case you have such legacy scripts it can be easier to replace uvm server rather than rewrite scripts.
   

As known, uvm report server implemented as singleton (singleton pattern at wiki). Using singleton is required for storage of one base of messages(namely, counters for each message type, dictionary of message [id], etc.).

uvm_report_server is implemented in class uvm_report_global_server (view source of ovm_report_server.svh).
Pay attention to static ovm_report_server global_report_server = null; and constructor if (global_report_server == null) global_report_server = new;

Let's illustrate how to override uvm_report_server: lets colour all errors and fatal errors in red:

1. So we create coloured_report_server inheriting from uvm_report_server and override virtual compose_message which does the actual formatting (also there is virtual summarize function which is called at the end of simulation and prints message stats, you can override it as well)
2. create coloured_report_server object and set reference to it in uvm_report_global_server.

import uvm_pkg::*;

`define BLACK(str)   {`"\033[30m`",str,`"\033[0m`"}
`define RED(str)     {`"\033[31m`",str,`"\033[0m`"}
`define GREEN(str)   {`"\033[32m`",str,`"\033[0m`"}
`define YELLOW(str)  {`"\033[33m`",str,`"\033[0m`"}
`define BLUE(str)    {`"\033[34m`",str,`"\033[0m`"}
`define MAGENTA(str) {`"\033[35m`",str,`"\033[0m`"}
`define CYAN(str)    {`"\033[36m`",str,`"\033[0m`"}
`define WHITE(str)   {`"\033[37m`",str,`"\033[0m`"}

`define BOLD(str)    {`"\033[1m`",str,`"\033[0m`"}

class coloured_report_server extends uvm_report_server;

    function string compose_message(
            uvm_severity severity,
            string name,
            string id,
            string message,
            string filename,
            int    line
      );
        string result;
        if ((severity==UVM_ERROR) || (severity==UVM_FATAL))
            result = super.compose_message(severity, `RED(name), `RED(id), `RED(message), filename, line);
        else
            result = super.compose_message(severity, name, id, message, filename, line);
        return result;
    endfunction
//you can overload virtual function void summarize(UVM_FILE file=0);
endclass

module coloured_server_test;
   initial begin
     coloured_report_server new_server;
     uvm_report_global_server glob;
     glob = new();
     new_server = new();
     glob.set_server(new_server);
   end

   initial begin:printing_block
     #1; //delay is needed for the first initial to finish
     uvm_report_info("info",$sformatf("info from: '%m'"));
     uvm_report_error("error-id", "printing in red"); 

     //uvm_run_test();

   end
endmodule

To run the snippet one should compile it with uvm support enabled. Most simulators support -uvm options which will autocompile the package and link all required libraries.

$mysimulator -uvm coloured_report_server.sv

Log output should look like this (header and summary omitted for clarity):


 UVM_INFO @ 1: reporter [info] info from: 'coloured_server_test.printing_block'
UVM_ERROR @ 1: reporter [error-id] printing in red


PS. It's worth mentioning that colorizing is inserting of escape sequences into output log. This may affect the grepping result of log. In case of info message we will be able to find it:
$ grep 'reporter \[info\]' simulation.log --> will find the info
but in case we are trying to find error-id with brackets the error won't be found as line turned into [31mreporter [0m [ [31merror-id [0m] [31m
$ grep 'reporter \[error-id\]' simulation.log --> won't find error-id

Install eclipse and systemverilog sveditor without root

SVeditor is a free opensource systemverilog editor plugin for eclipse.

I prefer to install eclipse manually, i.e without package managers.
The key reason is eclipse is being released more often than my operating system, Fedora, and fedora guys cease providing fresh packages for distribution in a year after release. Moreover, some servers and pc in our company use aged centos. So updating eclipse and its components from time to time without root password is crucial.

1. Download tar-gz eclipse release for appropriate arch(i686 or x86_64) at http://www.eclipse.org/downloads). I prefer  'Eclipse IDE for C/C++ Developers' package.
2. untar it. by itself eclipse is already available to be started (java-1.6.0-openjdk package must be installed).
3. download sveditor plugin named something like sveditor-0.6.1.ea08.jar.
   All downloads are available at http://sourceforge.net/projects/sveditor/files/sveditor/
   
   I found it useful to subscribe for fresh releases with RSS feed. Just follow this url to receive notification about new releases http://sourceforge.net/api/file/index/project-id/230781/mtime/desc/limit/20/rss
   
   By the way, sveditor's author is quite responsive to bug reports. It's possible to fire a bug anonymously at http://sourceforge.net/tracker/?group_id=230781&atid=1081015
4. if you can't set up eclipse to use proxy the actual plugin installation is a bit tricky. So go eclipse->help->install_new_software.
   Hit Add... button, in 'Location' field point full path to download 'sveditor...jar' file. 'Name' field may be left empty.
   After hitting Ok following dialog appears:
   
   
   check the needed boxes, next, next, accept sveditor license, next, restart.
   
   PS. After installing this and other plugins it's possible to set read-only access and share eclipse via NFS so colleagues could skip the installation part and just use it.

Using eclipse for functional verification projects

I use eclipse for following features:

1. highligthning, autocomplition, open declaration for
• make
• asm, c/c++,(systemc, systemc_verification sources)
• systemverilog (and ovm,uvm,vmm) (with sveditor plugin)
• python(with plugin)
2. integration cvs & svn
3. my hotkeys:
   
   • Ctrl+shift+R - open resource(read file) while typing its name. When project becomes huge (it usually does), it becomes unbearable to remember filesystem paths of all files. navigate->'open resource' is where eclipse beats simple editors like KDE kate.
   • F3 - open declaration
   • Ctrl+space - autocompletion 
   • Ctrl+D - delete line (yes, no more home,shift+end,delete)
   • Alt+up/down  - move line up/down ( yes, no more home, shift+end,ctrl+x,up,up,up,ctrl+v)
   • Alt+Shift+up/down - duplicate line
   • Ctrl+pageup/down - move to next tab 
   • Alt+left/right - jump to last edited place

Configurable pull-up, pull-down wires inside interface

During phy interface verification it mighty be handy to change pull-up (pull-down) resistors dynamically, i.e without recompiling verilog source codes and restarting simulation.

To implement this we need to assign second driver with weak strength to target wire. Depending on local variable this second driver will set logical 0 or 1.

Here is a workable systemverilog example of interface with configurable pull-ups and simple testbench with it.

// importing OVM here just for log messages
import ovm_pkg::*;

interface conf_pull_iface(
              inout wire clk,
              inout wire data);


    // using enumerations allows to view
    // enumeration names in waveform viewers instead of
    // integer values
    typedef enum {
        PULL_NONE=0,
        PULL_UP=1,
        PULL_DOWN=2
    } pull_mode_t;

    pull_mode_t pull_clk = PULL_NONE;
    pull_mode_t pull_data = PULL_NONE;

    logic clk_out;
    bit   clk_out_enable;

    logic data_out;
    bit   data_out_enable;

    assign clk = clk_out_enable ? clk_out: 'z;
    assign (pull1,pull0) clk = (pull_clk == PULL_UP) ?  1'b1:
                                ((pull_clk == PULL_DOWN) ? 1'b0: 'z);

    assign data = data_out_enable ? data_out : 'z;
    assign (pull1,pull0) data = (pull_data == PULL_UP) ?  1'b1:
                                 ((pull_data == PULL_DOWN) ? 1'b0: 'z);

    function void set_pull(int unsigned pull_clk_,
                           int unsigned pull_data_);

        assert(pull_clk_ <= PULL_DOWN);
        assert(pull_data_ <= PULL_DOWN);
        pull_clk = pull_mode_t'(pull_clk_);
        pull_data = pull_mode_t'(pull_data_);
        ovm_report_info("", $sformatf("%m: changed: pull_clk=%s, pull_data=%s",
                pull_clk.name(),
                pull_data.name()));
    endfunction

    function void set_enable(bit clk_enable,
                             bit data_enable);
        clk_out_enable = clk_enable;
        data_out_enable = data_enable;
        ovm_report_info("", $sformatf("%m: set enable: clk_enable=%s, data_enable=%s",
                clk_out_enable,
                data_out_enable));
    endfunction

endinterface


module tb;
    wire clk;
    wire data;

    conf_pull_iface iface(.*);

    initial begin
        #1ns iface.set_pull(1,2);
        #1ns iface.set_pull(2,1);
        #1ns iface.set_pull(0,0);
    end

endmodule

Using OVM-aware simulator we can run example:
$ mysimulator -ovm  configur_pull_ups.sv

Which would result in following log:
OVM_INFO @ 1: reporter [] tb.iface.set_pull: changed: pull_clk=PULL_UP, pull_data=PULL_DOWN
OVM_INFO @ 2: reporter [] tb.iface.set_pull: changed: pull_clk=PULL_DOWN, pull_data=PULL_UP
OVM_INFO @ 3: reporter [] tb.iface.set_pull: changed: pull_clk=PULL_NONE, pull_data=PULL_NONE


A couple of notes to mention:
1) we use enumerations to facilitate debugging, variables pull_clk and pull_data of enumeration type pull_mode_t will be shown in waveform viewer with appropriate string values (not plain int values). Moreover, we use pull_clk.name() function to print neat log messages.

2) we can't use enumerations in set_pull function, as this type can't be accessed outside of interface. So we add assertions to ensure passed int values are in enumeration range.