Re: Running large DB's on FreeBSD

On Fri, Oct 27, 2006 at 10:06:30AM -0400, Bucky Jordan wrote:
-----Original Message-----
From: owner-freebsd-stable@xxxxxxxxxxx [mailto:owner-freebsd-
stable@xxxxxxxxxxx] On Behalf Of Jim C. Nasby
Sent: Friday, October 27, 2006 12:44 AM
To: David Magda
Cc: Mike Jakubik; stable@xxxxxxxxxxx
Subject: Re: Running large DB's on FreeBSD

On Mon, Oct 23, 2006 at 08:15:04PM -0400, David Magda wrote:
As for Postgres on FreeBSD, FlighAware seems to be using it some
some
decent amount of data:

. Receiving the data and processing it puts them about 6 minutes
behind real time
. Generating one map can be done in about 160 milliseconds of CPU
time
. Capable of generating several million maps a day
. About 1 TB of stored data
. Approximately 40 million position updates on air craft per day

http://joseph.randomnetworks.com/archives/2006/05/12/flightaware-
freebsd-and-postgresql/

And that's on a dual opteron with 12G of memory and a run of the mill
RAID10 (for the database that is).

Yes.. but how many disks (size/type/rpm?) are in that RAID 10? I'm
guessing it's an external enclosure...

Also, I know 10k rpm vs 15 doesn't make much of a difference for
sequential, but random IO seems to be significantly improved. Granted,
it's not as dramatic as adding more spindles...

IIRC it's a 6 drive array of SATA. Nothing all that fancy.

I think the other point that may be relevant is the active section of
the data that you're accessing, and how good your design is in terms of
being able to access that directly. You could have a 1TB database, but
only have a portion that is frequently accessed/updated. In that case,
you might just need lots of storage, which is fairly inexpensive these
days. Also, your money might be better spent on more RAM- if you can fit
most of the active data in memory, that will also have a positive impact
on performance.

As pointed out, 10GB isn't really that much, especially when you can buy
relatively inexpensive servers with 8 or 16 GB of ram. Fitting over half
your db in memory is quit a luxury.

Well, what's most important is your system architecture. If you have a
poor design to start with, you'll never get good performance out of it.
--
Jim C. Nasby, Database Architect decibel@xxxxxxxxxxx
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"
_______________________________________________
freebsd-stable@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • NVIDIA FreeBSD kernel feature requests
    ... NVIDIA has been looking at ways to improve its graphics driver for the ... FreeBSD i386 platform, as well as investigating the possibility of adding ... and user mappings of I/O memory, ...
    (freebsd-hackers)
  • RE: FreeBSD 4.9 freezes
    ... Copyright 1992-2003 The FreeBSD Project. ... Timecounter "TSC" frequency 2793012088 Hz ... Hyperthreading: 2 logical CPUs ... avail memory = 1546452992 ...
    (freebsd-questions)
  • Re: Question about top values on memory usage
    ... Another question is that is httpd uses threads (as provided by FreeBSD) ... SIZE is the total amount of virtual memory that a process has allocated. ... RES is the amount of physical memory in use by the process and will only include that part of a process's virtual memory space which is currently allocated in physical memory. ... The amount of swap space will place a hard upper bound on the number of httpd processes you can run before sbrk start failing. ...
    (freebsd-stable)
  • Re: Question about top values on memory usage
    ... Maybe someone with deeper knowledge of the internals of FreeBSD ... As you see, 'size' is the same for all processes, while RES varies. ... the real memory taken by a process is RES and SIZE ... RES is the amount of physical memory in use by the process and will ...
    (freebsd-stable)
  • Re: [OT]Re: malloc
    ... goose wrote: ... Not *all* typical desktops perform the same; ISTR that FreeBSD ... and it _still_ ties up every single page of virtual memory ... to you could have googled for "FreeBSD overcommit ...
    (comp.lang.c)