As people are asking a lot about the workstation on my Instagram I decided to share our CG & VFX Production equipment specs and run through everything we have in our machines. We built them ourselves and there were no A to Z articles or lists of components available which would be really helpful if this was one of your first high-end builds. We've collected useful information by pieces and built our workstation in our best interest.
You might think why such a workstation is needed in a first place. Well... It's a wide topic to cover but in a few words traditional 3D rendering utilizes CPU which is powerful but not as fast as we sometimes would need to carry on a task. We all played computer games which look pretty good and are rendered instantly, so there should definitely be a way to utilize your Graphics Card performance for 3D CG & VFX tasks. There are numerous render engines available on the market, but we are using Octane Render Engine, world's first and fastest GPU-accelerated, unbiased, physically correct renderer. The Octane Render Engine performance scales linearly, so doubling the amount of GPUs will double the render speed.
There are ways to put more than 4 GPUs in one machine using clusters, but in this article we will mainly review our particular builds that we use in our projects. We prefer divide GPUs in several machines: Main workstations (Masters) and Render units (Slaves).
The Main Workstation build:
Let's look closer at our water-cooling loop, as it might be tricky to estimate the fittings needed right away at the planning stage. In order to get it right, you can use our list as a guide:
11x Black Chrome XSPC G1/4" to 7/16" ID, 5/8" OD Compression Fittings
These are fittings that will connect your radiators with a pump and water blocks
2x Bitspower G1/4" 90 Degree IG 1/4" Adapters
These two are needed for the bottom radiator as there is no space to use regular fitting. You would have to put your regular fittings on top of these to avoid any tube kinks.
3x XSPC G1/4" Male to Male Rotary Fittings
3x XSPC G1/4" Male to Female Rotary Fittings
These 6 fittings (3 when assembled together) will connect your GPU water blocks to each other. I must admit this was the trickiest part to figure out. I kept ordering different fittings which I assumed would fit, but they didn't. Finally, I tried to combine two fittings and they worked perfectly.
1 meter Clear XSPC 7/16" ID High Flex tube
If you have all the parts and are ready to build your monster, than this video is your beginners guide on how to build a PC. And This one will help you build your water loop.
Apart from that you will need couple of Molex extenders and Fan splitters. You might want to install additional LED lights. We used XSPC Twin 3mm LEDs for GPUs and CPU water blocks and NZXT HUE+ LED Controller & Strips. Fans are also lit in white.
For the eight fans installed on radiators we use NZXT Grid V2 control board and Lamptron CF525 for six others. Manual fans control becomes handy when you're not working on anything render-heavy. As you might have guessed the machine with 14 fans is quite noisy, so if you set the power to minimum when intense cooling is not needed, it will make it fairly quiet.
GPUs Idle temperature in our machine is around 22°C with a 30% fans speed. Under the heavy load GPUs temperature can go up to 45°C. In comparison to our render farm,
air-cooled GPUs can get heated up to 85°C under the heavy load. So water cooling is doing its job and cools down GPUs for at least 30°C. The liquid in the loop is around 25°C in idle and up to 35°C when intensively cooling down the system.
The temperatures of GPUs and water in the system may also vary depending on the ambient temperature. We performed a test in a room with 20°C. I left the machine idle for the night and took a screenshot next day, then started rendering a heavy sequence with 5000 samples/frame which heated GPU up to 42°C. I couldn't make it hotter, but sometimes, with particularly heavy rendering and simultaneous work in other applications (Especially CPU demanding), GPUs temperature can go up to 50°C and liquid heat up to 40°C due to an additional heat generated by CPU. That's why some builds have separated water loops for CPU and GPU cooling.
The Render Farm units build
As already mentioned we have render farm units that fit 3 GPUs each that linearly increase our render performance. We are using them for final renders which are always time-demanding, so additional power is very helpful to speed things up.
You only need Windows and Octane Standalone Slave daemon software installed and running. Everything is calculated by Master machine (M), and render units (S) serve as power horses.
The reason we are considering only 3 GPUs in a main station is that GTX1070 serves as Windows data output card. When you're rendering and all your cards are fully loaded, the whole system can become glitchy, as it struggles to output all Windows data and render at the same time. In our case, we have three screens connected, hence it becomes very handy to have spare GPU for Windows output. This way the whole system is always working flawlessly, even with all other GPUs fully loaded, thus you can keep Modeling, Animating or doing post-production tasks while rendering in the background.
To summarize, let's run through the vital elements that influence the performance in GPU powered CG & VFX production:
- Graphic cards. There are plenty of different GPUs available on the market with prices varying from £200 to £5000, but in our case the higher price doesn't always mean better. This is a super wide topic to discuss, but what we can definitely be sure about is that Nvidia are the leaders with their GTX1080, GTX1080Ti, GTX Titans and upcoming GTX2080 Graphic Cards at a very affordable price.
- Cooling. Did you know that your GPU can lost up to 40% of its performance by getting too hot? Well, now you know. Therefore consider powerful fans or some water for your build, to take the most out of it.
- Case. Cases are important because of their designs and airflow which affects your cooling effectiveness, so choose wisely.
All other PC elements do not affect GPU powered rendering as much as highlighted points.
Keep in mind that any PC can be planned as an ongoing project. You can always add and/or upgrade parts as you go. I wish you a good luck engineering your work horses.