Looking for speed increases in "make index" and pkg_version for ports

I have been thinking a lot about looking for speed increases for "make index" and pkg_version and things like that. So for example, in pkg_version, it calls "make -V PKGNAME" for every installed package. Now "make -V PKGNAME" should be a speedy operation, but the make has to load in and analyze bsd.port.mk, a quite complicated file with about 200,000 characters in it, when all it is needing to do is to figure out the value of the variable PKGNAME.

I suggest rewriting "make" so that variables are only evaluated on a "need to know" basis. So, for example, if all we need to know is PKGNAME, there is no need to evaluate, for example, _RUN_LIB_DEPENDS, unless the writer of that particular port has done something like having PORTNAME depend on the value of _RUN_LIB_DEPENDS. So "make" should analyze all the code it is given, and only figure it out if it is needed to do so. This would include, for example, figuring out .for and .if directives on a need to know basis as well.

I have only poked around a little inside the source for make, but I have a sense that this would be a major undertaking. I certainly have not thought through what it entails in more than a cursory manner. However I am quite excited about the possibility of doing this, albeit I may well put off the whole thing for a year or two or even forever depending upon other priorities in my life.

However, in the mean time I want to throw this idea out there to get some feedback, either of the form of "this won't work," or of the form "I will do it," or "I have tried to do this."

Best regards, Stephen
_______________________________________________
freebsd-hackers@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: Looking for speed increases in "make index" and pkg_version for ports
    ... So for example, in pkg_version, it calls "make -V PKGNAME" for every installed package. ... Now "make -V PKGNAME" should be a speedy operation, but the make has to load in and analyze bsd.port.mk, a quite complicated file with about 200,000 characters in it, when all it is needing to do is to figure out the value of the variable PKGNAME. ... Anyway, here is a multithreaded program "all-depends-list" that can get you double the speed on dual processor systems, and even some small speed gains on single processor systems. ...
    (freebsd-hackers)
  • Re: Looking for speed increases in "make index" and pkg_version for ports
    ... value of the variable PKGNAME. ... speed on dual processor systems, and even some small speed gains on ...
    (freebsd-hackers)
  • Re: Looking for speed increases in "make index" and pkg_version for ports
    ... So for example, in pkg_version, it calls "make -V PKGNAME" for every installed package. ... Now "make -V PKGNAME" should be a speedy operation, but the make has to load in and analyze bsd.port.mk, a quite complicated file with about 200,000 characters in it, when all it is needing to do is to figure out the value of the variable PKGNAME. ... Anyway, here is a multithreaded program "all-depends-list" that can get you double the speed on dual processor systems, and even some small speed gains on single processor systems. ...
    (freebsd-hackers)