понедельник, 6 августа 2012 г.

Directory layout showcase for simple verification

Lets show a directory layout for a little imaginative rtl-project with verification testbench. Here is a bit of terminology:

DUT - design under test, the RTL project we are going to verify.

Harness - it's a module where DUT plus all verification agents are instantiated.

Verification environment(VE) - harness plus scripts required to compile harness, run simulation, run post-processing, dump and view waveforms, run one testcase in semiautomated way multiple times, etc. Writing such scripts may seem tedious but it's a must. Verification engineer developing testcases shouldn't worry about how the RTL is actually compiled/elaborated. 'make vlog sim SEED=1' knowledge is enough for testcase developer and RTL designer to run the test. Development of VE is verification integrator's job. It does require good knowledge of scripting languges.

Testcase(tc) - actually the test verification engineer is working on. Each testcase must have the same rules to compile, run. All testcases must print a unique tail with info about whether testcase passed or failed.

Teststep - testcase must be preferably splitted on a few steps. Having scores of testcases is difficult to maintain, document and track. So dealing with testcase with 3-5-7 teststeps united by common features is ok for verification engineer. On the other hand it drastically reduce number of testcases when we overview the test plan with list of testcases.

Testbench(tb) - verification environment + all testcases for DUT verification. So for verification developer testbench is a place where he/she is developing/working/running_simulation. For verification manager/team leader testbench is a final product, i.e 'RTL team is doing RTL project, verification team is doing testbench for it'.

Now get down to layout itself. The source code (directories and makefiles) is located in my repo at bitbucket and can be downloaded at tar-archive here

So the directory layout is in the picture below.


As one can see the layout reflects the DUT, harness, environment (makefiles are the scripts) and testbench.
Some important notes:

* project scripts must use relative paths. It's an inevitable requirement for multiuser, multihost environment. No 'export PROJECT_HOME=/home/myuser/projects/myproject'. Once testcases are all at the same level of hierarchy makefiles easily allow to use relative paths.

* rtl-designers, verification integrator, testcase-verificators are working in different directories on different files. Never rtl-designer should edit testbench library or tests inside testbench directory, as well as never verificators should edit files in RTL directory.

* extending idea of separation, testbench library can contain verification IPs/packages that can be considered mature and widely used in other projects. This means these packages must not be edited. All project-specific agents are going into harness directory (or harness/tb_lib in case there are a lot of such agents).

* more extending idea of separation, all testcases are located in separate directories, in case one tc-developer mistakenly commits broken testcase (non-added file or just missing semicolon), other devs won't notice it (regression will, you have it, right? :)

* having her/his own workspace (literally testcase directory) verification engineer is free to create new scripts/makefile targets, still not interfering with other scripts/make_targets from other engineers

* it becomes rather easy to create regression script to run all testcases in testbench. For example (run in testcase directory):
$ for k in ../tc_*; do echo running $k; pushd $k; make|grep -i error; popd; done

* what we do sacrifice is place(path) where we can call scripts (compilation, simulation), i.e simulation can only be started at two levels down to PROJECT_HOME

* one can say that there is extra work for harness compilation inside each testcase directory and will be right! First of all, it's extremely easy to create symlink to compiled snapshot in each testcase directory (again, it can be a make target). So it's easy to switch from one approach to another. Second, imagine I'm developing two testcases. One is finished and I want to compile harness with enabled optimization and run multiple times. The second testcase in an early stage development and I debug it with dumping waveforms. In case two testcases were located in one directory then:
  1. it would require having two differently compiled snapshots (hence compilation scripts would fattened with variables)
  2. running two simulations with different log-files in one directory is a nightmare.
  3. developing and debugging scripts that would organize such scenario become difficult.