Use jemalloc where available for ~6% performance improvement

[Majora320]

Description

jemalloc is an alternative allocator that provides better performance than the default glibc malloc in most situations, especially for memory-heavy applications. This PR uses jemalloc by default when available and adds it to the default install lists.

Motivation and Context

vtr is slow. Anything that shaves a few seconds off each iteration is useful.

How Has This Been Tested?

I tested jemalloc locally on a 7800X3D along with tcmalloc and micalloc, two other popular allocator options. The following command was used:

/usr/bin/time -v vpr ../arch/titan/stratixiv_arch.timing.xml ./benchmarks/titan_blif/titan23/stratixiv/neuron_stratixiv_arch_timing.blif --sdc_file ./benchmarks/titan_blif/titan23/stratixiv/neuron_stratixiv_arch_timing.sdc --max_router_iterations 400     --router_lookahead map     --initial_pres_fac 1.0     --router_profiler_astar_fac 1.5     --seed 3     --route_chan_width 400 -j8

Here is a table of times and relative improvements.

	malloc version	vpr time (s)		vpr max_rss (MiB)		user time (measured by time)		time rss (kbytes)
glibc malloc	2.42	123.62		3160.3		158.05		3236160
tcmalloc	4.6.3	117.68	-4.81%	3072.8	-2.77%	149.14	-5.64%	3148120	-2.72%
mimalloc	2.1	118.06	-4.50%	3818.4	20.82%	148.99	-5.73%	3910056	20.82%
jemalloc	5.3.0	117.26	-5.14%	3042.9	-3.71%	148.32	-6.16%	3115924	-3.72%

As you can see, jemalloc shaves about 6% off of total execution time and reduces peak memory usage by about 4%.

Please let me know if any documentation changes are required as a result of this change. Thanks :)

Types of changes

• Bug fix (change which fixes an issue)
• New feature (change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• My change requires a change to the documentation
• I have updated the documentation accordingly
• I have added tests to cover my changes
• All new and existing tests passed

[AmirhosseinPoolad]

vtr is slow.

We're trying our best :'( Although I gotta say the last time anyone checked, VtR was actually ~30% faster than Quartus. FPGA CAD is just that hard.

I personally like the implementation here, not super intrusive and nothing bad happens if jemalloc is not installed on user machines. Approved the CI tests to see if everything works as expected. Can you test the whole Titan benchmark suite to see gains on a more diverse set of circuits? You can run them using:

vtr_flow/scripts/run_vtr_task.py regression_tests/vtr_reg_nightly_test2/titan_quick_qor -j[cores]

The run_vtr_task script outputs a file in vtr_flow/tasks/regression_tests/vtr_reg_nightly_test2/titan_quick_qor/run[num] called parse_results.txt or something. You can copy that into google sheets to get all the relevant numbers. Please also check memory usage. I don't expect meaningful changes in large circuits, but the new allocator probably gets memory pages from the OS in a different way and I'm seeing some CI failures in smaller tests because of higher memory usage.

They do take a while to run (sorry!) and I would probably not put in -j8 because of memory usage. I think dividing your RAM by 8 would give you a fairly safe number of benchmarks you can run at the same time.

[AlexandreSinger]

Looks like one testcase uses more memory and is causing a QoR failure:

We put bounds on our QoR metrics to keep noise down, so it is possible that this is overall good for memory usage and runtime, and this one case is a fluke.

I agree with @AmirhosseinPoolad here that we need to see the results on more testcases to bring something like this in.

[stefanpie]

I proposed something like this in #3189. I continued down this road a bit further but never finished the project at that time.

I wanted to note that, when expanding testing to different test cases, you should be sure to test different VTR modes. I found large runtime improvements when no routing channel size was specified and VTR searched for it automatically. However, when I set a fixed channel size and it ran one complete flow, the runtime improvements were greatly diminished. This is obvious in hindsight, since VTR is basically relaunching parts of the flow over and over again with some kind of reallocation each time. But I tripped over obvious things like this when I first went down this path.

I also was not sure if there is a clear tradeoff between runtime and memory usage that the VTR developers would prefer. For example, how much memory overhead is acceptable for a given runtime speedup? I assumed that runtime improvements far outweigh any memory overhead, but I have not used VTR in any industrial use cases where even the baseline memory usage is quite large. Any insight into this would definitely be interesting.

[vaughnbetz]

The two regtest failures are not a real concern; they are small QoR changes (memory footprint) on the same tiny design. They can be resolved by updating the golden results.

They key thing we'd need to merge this would be a full QoR run as @AmirhosseinPoolad noted. If you don't have a machine that can run that, perhaps Ayan could help.

[vaughnbetz]

The code changes look minimally intrusive and hence fine.
A quick web search shows jemalloc isn't supposed to support valgrind; not sure how you've worked around this (turned off jemalloc for some builds?) as we do use valgrind, and wouldn't want to lose it. If this implicitly makes valgrind not work, we'd need to figure out a solution to that too.

So outstanding list is:

• Bigger QoR experiment
• valgrind status / interaction
• update golden results on the two failing tests

[gigeresk]

We should also disable jemalloc when VTR_ENABLE_SANITIZE is set, as ASan also replaces malloc

[gigeresk]

I ran valgrind on the jemalloc build and did not run into any issues; all the information seems to have been preserved. I'd need a +1 on this, though, since I've used ASan more than I've used valgrind...