Do not build debug archs on tests

There is sufficient added complexity to manage this, for limited value.
If there are issues with tests failing that require debug symbols, one
can manually modify the pytest-generated wrappers to add the symbols
back.