Compile Times...

Compile Times...

Re: Compile Times... Posted by Riven on Tue Mar 27th 2007 at 4:08am
Riven
1640 posts
Posted 2007-03-27 4:08am
Riven
Wuch ya look'n at?
super admin
1640 posts 1266 snarkmarks Registered: May 2nd 2005 Occupation: Architect Location: Austin, Texas, USA
Hey guys, I'm looking to upgrade my system soon...

All I want to know is which choice of hardware would have the greatest effect on my compile times? I looked around online and in the forums, but didn't find anything too specific on what was best for speeding up compiling. (one thread sugested the RAM was paramount, but I want to find out for sure...)

Here is my current system:

ASUS A8n SLI premium (socket 939)
OCZ 2GB DDR 400 (PC 3200)
AMD 2.4GHz 3700+
Nvidia 7800 Gt 256MG
2x WD 250 SATA HDD (both backups)
1x WD 250 IDE HDD (main; os install)

Here are my choices:

exchange for>AMD Athlon FX-57 San Diego 2.8 GHz 1MB L2 Cache
exchange for>AMD Athlon 64 X2 4600+ Toledo 2.4GHz 2 x 512KB L2 Cache (dual-core)
add to>RAM: 2GB DDR 400 (PC 3200)

I'm limited to my socket 939; these were the choices that best fit my budget IMO, and I believe those processors are the best that is left available for the 939.

Any help here would be greatly appreciated guys! (Not to mention informative).
Blog: www.playingarchitecture.net
LinkedIn: Eric Lancon
Twitter:@Riven202
Re: Compile Times... Posted by Gorbachev on Tue Mar 27th 2007 at 4:47am
Gorbachev
1569 posts
Posted 2007-03-27 4:47am
1569 posts 264 snarkmarks Registered: Dec 1st 2002 Location: Vancouver, BC, Canada
Honestly your current system isn't bad. I really only know of the older compiler and it's been a while...I used to know it inside-out almost, but I haven't even looked at HL stuff in over a year. There's a max that certain amounts of RAM and Processor speed will cap at due to different factors, although if the compilers can take advantage of dual-cores then that's probably the only thing I can think of that will make a marked difference in your case.
Re: Compile Times... Posted by Crono on Tue Mar 27th 2007 at 5:39am
Crono
6628 posts
Posted 2007-03-27 5:39am
Crono
super admin
6628 posts 700 snarkmarks Registered: Dec 19th 2003 Location: Oregon, USA
Memory speed and CPU speed are the only things that predominantly effect compile times (HDD access speed is also a factor)

I don't see why you'd want to do an insane-blow-my-brains-out-upgrade, really. I could see upgrading the CPU to a dual core, or something like that, but doing something extravagant wouldn't be prudent when talking about cost.

People seem to be asking this question a lot, so, I'll lay down a couple things to remember:

1) You should be more concerned with how well the map runs, not how long you'll be inconvenienced for when you compile it. Also be aware that while you're building the level you should be using "fast" compile options. Then when you make a final release, you turn all the "bells and whistles" on, so to speak.

2) It's a compiler. All operations being done are pre-computations or translations so the game can read in the map. So, everything is done on the CPU. You can treat it like any cpu intensive task. It has no comparison to how well the actual game runs on your system.
People seem to be confused by this for some reason.

Foremost: What is happening (most likely) is that the CPU can't possibly compute the data it's working on faster than the data can be retrieved from memory.

We can pretty much assume that the calculations being done (on vis and lighting, for example) are far too complex for the system bus to be slowing the process down by leaving the CPU idle. So, the only way you'll notice ANY speed up is if you get a faster (or multi-core) CPU.

As for the amount of memory you have, there's a good chance that Windows forces Hammer (or whatever editor) to use virtual memory even if physical memory is not even close to full ... so adding more memory to 2GB (which is a good chunk of memory, twice as much as myself) wouldn't do much of anything. You will definitely notice some increase in speed because not as many things will be paged, but for specifically speeding up compile times ... it'll have little to no effect, when you're compiling data, it's not large enough nor old enough to get the boot into VM (unless you begin compiling and then run a game or something!)

Overall, I wouldn't suggest upgrading anything really, especially if your goal is to simply cut down compile times. A better solution, would be to figure out how level complexity works in whatever game this is for (or language!) and then try to optimize the way you create things to suit that.
Blame it on Microsoft, God does.
Re: Compile Times... Posted by reaper47 on Tue Mar 27th 2007 at 8:36am
reaper47
2827 posts
Posted 2007-03-27 8:36am
reaper47
member
2827 posts 1921 snarkmarks Registered: Feb 16th 2005 Location: Austria
I would like to ask you: How long do compile times take for you now? If you're talking about multiple hours (or more than 5 minutes for vvis), then the only way to really speed up things is optimizing the brush geometry.

People suggesting RAM as the most important part for compiling is IMO a relic from the times when 64 or 128 MB were standard (HL1 times) and RAD used the hard-drive which was really slow. I've never seen any compile program using considerably more than 170 MB of RAM. 4GB would be total overkill.

I think the compiler can handle multiple cores which sounds like it could speed up things quite a bit. But I don't know.

Again, if you are looking for a way to speed up those 8-hour compiles you're probably looking at the wrong place :smile:
Why snark works.
Re: Compile Times... Posted by wil5on on Tue Mar 27th 2007 at 9:28am
wil5on
1733 posts
Posted 2007-03-27 9:28am
wil5on
member
1733 posts 570 snarkmarks Registered: Dec 12th 2003 Occupation: Mapper Location: Adelaide
The general rule for compile times is leave it overnight, if it still takes too long, your map is broken.
"If you talk at all during this lesson, you have detention. Do you understand?"
  • My yr11 Economics teacher
Re: Compile Times... Posted by Wild Card on Tue Mar 27th 2007 at 11:45am
Wild Card
2321 posts
Posted 2007-03-27 11:45am
2321 posts 391 snarkmarks Registered: May 20th 2002 Occupation: IT Consultant Location: Ontario, Canada
Yup, agreed with a lot here. Your rig is fine, short of being NASA super computer compatible :cool:

As far as I know, going to a slower dual core rather than a faster single core would slow down your compiling because compiling is still a single-process... err.. process.

But check your map-work. Do you have overlapped brushes? Do you have overly stretched textures? Do you have long brushes which get hit by different lighting entities? (If thats the case, cut the brush into smaller pieces). Maybe texture outside brushes (which face the void) with a null texture or something.

Then again, Im going by 2-3 year old HL1 data here but I'd imagine it still applies.
Re: Compile Times... Posted by Forceflow on Tue Mar 27th 2007 at 12:04pm
Forceflow
2420 posts
Posted 2007-03-27 12:04pm
2420 posts 451 snarkmarks Registered: Nov 6th 2003 Occupation: Engineering Student (CS) Location: Belgium
There is still no multi-core compiling, so multiple cores won't help.
Is there a netvis release for source compiling, actually ?
:: Forceflow.be :: Nuclear Dawn developer
Re: Compile Times... Posted by reaper47 on Tue Mar 27th 2007 at 1:03pm
reaper47
2827 posts
Posted 2007-03-27 1:03pm
reaper47
member
2827 posts 1921 snarkmarks Registered: Feb 16th 2005 Location: Austria
There is still no multi-core compiling, so multiple cores won't help.
Thanks for making that clear, I always wondered. Then what Wildcart said applies and multi-core processors could even slow down compiles as the single cores themselves are actually slower. Paradoxical! :o
Why snark works.
Re: Compile Times... Posted by Wild Card on Tue Mar 27th 2007 at 1:50pm
Wild Card
2321 posts
Posted 2007-03-27 1:50pm
2321 posts 391 snarkmarks Registered: May 20th 2002 Occupation: IT Consultant Location: Ontario, Canada
/me wonders how you got 'Wildcart' out of 'Wild Card'

Dual cores wont slow down a compile process. Dont get that idea. But buying a dual core at a slower speed rather than buying a single core at a faster speed will make for slower compiling compared to the latter.

But at the same time, with a dual core, you can pretty much safely compile your map and then play a video game. Provided you have enough (2gb I'd say) RAM.

I have a dual core at home with 2gigs, and although I've never used it to compile, I can safely have 2 video games going, a virtual machine with 384mb of RAM, and Folding@Home at 100% CPU.
Re: Compile Times... Posted by reaper47 on Tue Mar 27th 2007 at 3:03pm
reaper47
2827 posts
Posted 2007-03-27 3:03pm
reaper47
member
2827 posts 1921 snarkmarks Registered: Feb 16th 2005 Location: Austria
Embarrassingly enough that's a "fixed" typo from "Wildcat". Sorry, Wild Card, I should stop typing with my ass.
Why snark works.
Re: Compile Times... Posted by Wild Card on Tue Mar 27th 2007 at 3:10pm
Wild Card
2321 posts
Posted 2007-03-27 3:10pm
2321 posts 391 snarkmarks Registered: May 20th 2002 Occupation: IT Consultant Location: Ontario, Canada
Typing with your ass? No by all means continue, that sounds like quite a talent :biggrin:
Re: Compile Times... Posted by reaper47 on Tue Mar 27th 2007 at 5:16pm
reaper47
2827 posts
Posted 2007-03-27 5:16pm
reaper47
member
2827 posts 1921 snarkmarks Registered: Feb 16th 2005 Location: Austria
Yea, its goood traning.
Why snark works.
Re: Compile Times... Posted by Wild Card on Tue Mar 27th 2007 at 6:20pm
Wild Card
2321 posts
Posted 2007-03-27 6:20pm
2321 posts 391 snarkmarks Registered: May 20th 2002 Occupation: IT Consultant Location: Ontario, Canada
training*

:rolleyes:
Re: Compile Times... Posted by Forceflow on Tue Mar 27th 2007 at 10:39pm
Forceflow
2420 posts
Posted 2007-03-27 10:39pm
2420 posts 451 snarkmarks Registered: Nov 6th 2003 Occupation: Engineering Student (CS) Location: Belgium
One 3 Ghz core: regular compile time.
Two 1.8 Ghz cores making up a dual core, but only one can be used by the Source compile libraries, resulting in a slower compile time.

I know I'm over-simplifying the architecture of dual-core processors here (after all, today's processors are hardly judged by clock frequency), but that's basicly the idea Wild Card was stating.

On the record: don't upgrade your PC to compile faster. Really, unless you buy a small storage room full of Dell Poweredge Servers, it won't help. Optimize your maps.

If only Netvis was there. You could buy a bunch of cheap pentium II/III's, throw them in a lan, and have your own compile-bench ! (Sure, sure, it only works for vis ... but still .. the sheer coolness of it. dreams)

If you do have multiple computers at your disposal, you can let one handle the RAD process, one for VIS, ... it's a lot of file swapping, but it speeds up the overall process.
:: Forceflow.be :: Nuclear Dawn developer
Re: Compile Times... Posted by reaper47 on Tue Mar 27th 2007 at 10:43pm
reaper47
2827 posts
Posted 2007-03-27 10:43pm
reaper47
member
2827 posts 1921 snarkmarks Registered: Feb 16th 2005 Location: Austria
Re: Compile Times... Posted by Naklajat on Tue Mar 27th 2007 at 11:26pm
Naklajat
1137 posts
Posted 2007-03-27 11:26pm
Naklajat
member
1137 posts 384 snarkmarks Registered: Nov 15th 2004 Occupation: Baron Location: Austin, Texas
Source maps compile much faster than GoldSrc maps, especially if you use func_detail copiously (so do so if you don't already).

o

Re: Compile Times... Posted by mazemaster on Tue Mar 27th 2007 at 11:45pm
mazemaster
890 posts
Posted 2007-03-27 11:45pm
890 posts 438 snarkmarks Registered: Feb 12th 2002
Which part of the compile process takes the longest on your current setup?
http://maze5.net
Re: Compile Times... Posted by Riven on Wed Mar 28th 2007 at 12:08am
Riven
1640 posts
Posted 2007-03-28 12:08am
Riven
Wuch ya look'n at?
super admin
1640 posts 1266 snarkmarks Registered: May 2nd 2005 Occupation: Architect Location: Austin, Texas, USA
Well, in answer to Reaper47, my compile times aren't long at all really, and my mapping techniques are pretty solid (as far as optimization is concerned) I think I put into practice most of what they suggest to do to optimize a map (for in-game running, which now, thanks to Crono, I know is completely irrelevant to compile times.)

In answer to mazemaster: I would say the lighting (RAD) takes the longest, then VIS.

However, the AMD Athlon 64 X2 4600+ Toledo 2.4GHz dual-core processor that I have listed runs at the same clock speed that my 3700+ runs at. I find that if I installed that with a good heatsink, I could overclock that by at least .2-.3 GHz over. But I suppose I am at the best state now for my computer in compile times. The dual-core sounds most beneficial for being able to compile a map and complete other tasks simultaneously.

The reason stands because I am drawing up plans for some rather large maps, and would hope to have more time to fix problems within them rather than spending that time waiting on a compile process.

But yea, you could say it's all because of inconvenience. I might keep a look-out for better hardware choices to benefit my game design needs, but I'm backing down now, I suppose if I do buy anything it might be that dual-core, but I'll wait for prices to drop...

Thanks for the input!
Blog: www.playingarchitecture.net
LinkedIn: Eric Lancon
Twitter:@Riven202
Re: Compile Times... Posted by Crono on Wed Mar 28th 2007 at 12:37am
Crono
6628 posts
Posted 2007-03-28 12:37am
Crono
super admin
6628 posts 700 snarkmarks Registered: Dec 19th 2003 Location: Oregon, USA
This is what I was talking about earlier. The one that will always take the longest (in a full fledged map, thus situations where you care about compile times) is radiosity calculations. Calculating the bsp tree and vis leafs would not take as long, so unless you have a very poor layout (where backtracking would happen when constructing that tree) it will always be faster.

So, the tool that is your friend in hl2 is the cordon texture. Check it out if you're already unaware of it.
Blame it on Microsoft, God does.
Re: Compile Times... Posted by mazemaster on Wed Mar 28th 2007 at 8:38am
mazemaster
890 posts
Posted 2007-03-28 8:38am
890 posts 438 snarkmarks Registered: Feb 12th 2002
Alright then, so long as you aren't running out of RAM, I would say go with the beefier CPU.

The reason I asked about your current situation is that vis is notoriously stupid and so if that was the bottleneck then there are a lot of simple tricks that can speed it up considerably (15 hrs to 2 hrs, for example), and which you should do before buying new hardware.
http://maze5.net
Re: Compile Times... Posted by Mr.INSANE on Fri Mar 30th 2007 at 1:06am
Mr.INSANE
156 posts
Posted 2007-03-30 1:06am
156 posts 86 snarkmarks Registered: Jan 29th 2006 Occupation: Student Location: California,USA
There is still no multi-core compiling, so multiple cores won't help.
Is there a netvis release for source compiling, actually ?
Wrong, In compiling tools it allows you to choose how many thread's you want to use, Which can then be changed to witch core does what.

This is for Goldsrc with ZHLT

Generally if i wanted to make some sort of compiling monster id do the following

Single core (Honestly if its duel purpose compiling and mapping Duel Core is the way to go becuase you can still use one core while the other is compiling)
Make sure this is pretty damn fast Proccesor

Memory - The sky is the limit but i think You guys already said 2 Gb is good hell thats more than enough, Latency is important btw

The next is basicly me rambling about harddrive setups and other crap

id probally end up getting

Sata Western Digital Raptors in a raid configuration

(take in mind thats insane)
Why Do we all have to wear these ridiculous ties
Re: Compile Times... Posted by Naklajat on Fri Mar 30th 2007 at 1:35am
Naklajat
1137 posts
Posted 2007-03-30 1:35am
Naklajat
member
1137 posts 384 snarkmarks Registered: Nov 15th 2004 Occupation: Baron Location: Austin, Texas
INSANE
Sorry I couldn't resist :razz:

If you throw a bunch of beefy parts in your compy, like twice the RAM, some 10K RPM hard drives and a CPU that takes a whole lot more jiggawatts, don't neglect to upgrade your power supply. An overstressed power supply could end up frying itself and all your expensive new parts, and a cheap (low amperage, low efficiency) power supply adds a lot of heat and causes unstable voltages, overvoltage, and noisy current, all of which can lessen the lifespan of your components.

o

Re: Compile Times... Posted by Mr.INSANE on Fri Mar 30th 2007 at 2:53am
Mr.INSANE
156 posts
Posted 2007-03-30 2:53am
156 posts 86 snarkmarks Registered: Jan 29th 2006 Occupation: Student Location: California,USA
Just throw in a BFG 1kw power supply and youll be good
Why Do we all have to wear these ridiculous ties
Re: Compile Times... Posted by Crono on Fri Mar 30th 2007 at 5:02am
Crono
6628 posts
Posted 2007-03-30 5:02am
Crono
super admin
6628 posts 700 snarkmarks Registered: Dec 19th 2003 Location: Oregon, USA
Nothing needs anything close to a kW of power. Getting a kW power supply would be a waste of money. If you want to plan ahead, look at a 600W power supply.
Blame it on Microsoft, God does.
Re: Compile Times... Posted by Riven on Fri Mar 30th 2007 at 5:13am
Riven
1640 posts
Posted 2007-03-30 5:13am
Riven
Wuch ya look'n at?
super admin
1640 posts 1266 snarkmarks Registered: May 2nd 2005 Occupation: Architect Location: Austin, Texas, USA
I have a 550W that'll do eh? I was expecting to upgrade down the road so I got what I thought was a hefty power supply to begin with.
Blog: www.playingarchitecture.net
LinkedIn: Eric Lancon
Twitter:@Riven202
Re: Compile Times... Posted by Orpheus on Fri Mar 30th 2007 at 10:17am
Orpheus
13860 posts
Posted 2007-03-30 10:17am
Orpheus
member
13860 posts 2024 snarkmarks Registered: Aug 26th 2001 Occupation: Long Haul Trucking Location: Long Oklahoma - USA
I've always wondered. Does say a 500w tower pull 500w all the time or is it just capable of 500w peak?

Whats the idle wattage, if its not?

I have a 500w tower, I'd hate to think that 5 100w light bulbs are running 24/7. :sad:

The best things in life, aren't things.
Re: Compile Times... Posted by wil5on on Fri Mar 30th 2007 at 2:09pm
wil5on
1733 posts
Posted 2007-03-30 2:09pm
wil5on
member
1733 posts 570 snarkmarks Registered: Dec 12th 2003 Occupation: Mapper Location: Adelaide
Your PSU would get awfully hot if it were idling and still drawing 500W. I think the quoted wattage is maximum power output, so if you have 500W worth of stuff connected to it it will probably draw a bit more than that, but only slightly. Idle power drain from mains depends on idle power usage of specific parts in your box, it probably wouldnt be anywhere near 500W.

But if youre worried about wasting power, turn the thing off at night.
"If you talk at all during this lesson, you have detention. Do you understand?"
  • My yr11 Economics teacher
Re: Compile Times... Posted by Orpheus on Fri Mar 30th 2007 at 10:54pm
Orpheus
13860 posts
Posted 2007-03-30 10:54pm
Orpheus
member
13860 posts 2024 snarkmarks Registered: Aug 26th 2001 Occupation: Long Haul Trucking Location: Long Oklahoma - USA
I do, sometimes but I am oldschool and believe that more damage is done by turning them off/on.

Next question:

I have a mainboard that supports both ATA and SATA Hard drives. Can you use both simultaneously?

I ask because years ago I had a mainboard that supported both pc133 and pc2100. You had to chose which to use. You couldn't be running both at once.

I will be getting my library up soon and want to invest in a SATA drive and use my old 160gig ata drive for the library.

The best things in life, aren't things.
Re: Compile Times... Posted by Mr.INSANE on Fri Mar 30th 2007 at 11:28pm
Mr.INSANE
156 posts
Posted 2007-03-30 11:28pm
156 posts 86 snarkmarks Registered: Jan 29th 2006 Occupation: Student Location: California,USA
Nothing needs anything close to a kW of power. Getting a kW power supply would be a waste of money. If you want to plan ahead, look at a 600W power supply.
Shouldve added my /Sarcasm tags
Why Do we all have to wear these ridiculous ties
Re: Compile Times... Posted by mazemaster on Fri Mar 30th 2007 at 11:30pm
mazemaster
890 posts
Posted 2007-03-30 11:30pm
890 posts 438 snarkmarks Registered: Feb 12th 2002
"Nobody will ever need more than 640W of power."

-Crono
http://maze5.net
Re: Compile Times... Posted by fishy on Sat Mar 31st 2007 at 12:20am
fishy
2623 posts
Posted 2007-03-31 12:20am
fishy
member
2623 posts 1476 snarkmarks Registered: Sep 7th 2003 Location: glasgow
Orpheus said:
I have a mainboard that supports both ATA and SATA Hard drives. Can you use both simultaneously?
that would probably depend a lot on the individual board, but it works fine on my [now dated] nforce2
i eat paint
Re: Compile Times... Posted by Orpheus on Sat Mar 31st 2007 at 1:45am
Orpheus
13860 posts
Posted 2007-03-31 1:45am
Orpheus
member
13860 posts 2024 snarkmarks Registered: Aug 26th 2001 Occupation: Long Haul Trucking Location: Long Oklahoma - USA
Thanx Fisheye.

That's good news.

The best things in life, aren't things.
Re: Compile Times... Posted by wil5on on Sat Mar 31st 2007 at 2:38am
wil5on
1733 posts
Posted 2007-03-31 2:38am
wil5on
member
1733 posts 570 snarkmarks Registered: Dec 12th 2003 Occupation: Mapper Location: Adelaide
Orpheus said:
I do, sometimes but I am oldschool and believe that more damage is done by turning them off/on.
Thats silly, for the good of the planet, and your power bills, turn it off when youre not using it. I have machines that still work after being on 3-10 hours a day, every day, for 5ish years.
"If you talk at all during this lesson, you have detention. Do you understand?"
  • My yr11 Economics teacher
Re: Compile Times... Posted by Crono on Sat Mar 31st 2007 at 6:39am
Crono
6628 posts
Posted 2007-03-31 6:39am
Crono
super admin
6628 posts 700 snarkmarks Registered: Dec 19th 2003 Location: Oregon, USA
A lot of people ask me if they should get a 1kW power supply. They're not kidding either.
"Nobody will ever need more than 640W of power."
-Crono
I didn't say that. But currently and in the near future, 600W+ consumption seems really unlikely unless the parts you have, have the absolute worst circuit designs in existence.

Turning your computer off does not hurt it ... using your computer hurts it.

I think what you may be confusing it with is battery life. Turning a laptop repeatedly on and off, for example, will drain more power than leaving the thing on. It's because there's a high level of disc activity and all the memory is being "combed" and things like that. Lots of computations. While leaving the computer on will allow it to have idle state.

Then again, when it comes to laptops, the biggest power drainer is the screen.

Any board with a PATA and SATA connection will allow both simultaneously (they're seperate circuits). The question of obscurity is "can you use them both with RAID" and that's when you check the chipset manufacturer's documentation. Most recent ones allow that also.
Blame it on Microsoft, God does.
Re: Compile Times... Posted by Orpheus on Sat Mar 31st 2007 at 10:44am
Orpheus
13860 posts
Posted 2007-03-31 10:44am
Orpheus
member
13860 posts 2024 snarkmarks Registered: Aug 26th 2001 Occupation: Long Haul Trucking Location: Long Oklahoma - USA
wil5on said:
<DIV class=quote>
<DIV class=quotetitle>? quoting Orpheus</DIV>
<DIV class=quotetext>I do, sometimes but I am oldschool and believe that more damage is done by turning them off/on.
Thats silly, </div></div>

Perhaps, but it is a view shared by many old school types.

Technology changes. Once upon a time a screensaver was needed. That concept is no longer true. You can leave your monitor on indefinitely with no harm incurred.

Perhaps the same is now true with the tower.
However, I do not leave my pc on idle most nights. How do you reckon my library soared past 30 gigs in so short a time period? :wink:

The best things in life, aren't things.
Re: Compile Times... Posted by wil5on on Sat Mar 31st 2007 at 12:15pm
wil5on
1733 posts
Posted 2007-03-31 12:15pm
wil5on
member
1733 posts 570 snarkmarks Registered: Dec 12th 2003 Occupation: Mapper Location: Adelaide
I've heard of lower quality LCDs, and definitely CRTs not liking being left on all the time, but thats besides the point. You wont notice any lifetime differences between a box left on and a box thats turned off at night, at least not until theyre outdated. So the only consideration there is running cost, which is better than halved by not leaving it on constantly.

If its doing something useful overnight then fine, but wasted power is not a good thing. A lot of people (as far as I can see, only in the US) leave their computers on 24/7 doing absolutely nothing for most of that, which is just decadent. Think of the ice caps, turn it off when youre not using it.
"If you talk at all during this lesson, you have detention. Do you understand?"
  • My yr11 Economics teacher