HTSeq_2009_Module 6_2009.pptVIP

Canadian Bioinformatics Workshops www.bioinformatics.ca Data Management Asim Siddiqui Bioinformatics Workshop Next Generation Sequencing 26th July 2008 It’s a new world Next gen sequencers generated Gb of data Architecture matters Why should you care? Architecture matters If you are choosing the compute resources for your lab, designing the correct system architecture is important to get the most use out of the systems If you are using the compute resources of your lab, understanding the system architecture will help you get the most out of the system What you won’t learn. This lecture will provide you with a basic understanding of computer systems, but.... Consult your local expert If you are the local expert, make friends at a large genome centre ? The basics CPU Disk space RAM Bandwidth CPU The speed of your CPU determines how quickly it can process instructions Many bioinformatics operations fall into the “embarrassingly parallel” category Getting a results faster is as simple as adding more CPUs = clusters How many CPUs? RAM RAM is fast storage close to the CPU By loading data from disk to RAM, the CPU can execute instructions much more rapidly How much RAM? Typical sizing is 2GB of RAM per CPU This works fine for most aligners Assemblers typically need much more RAM If you don’t have enough RAM, the CPU will need to make use of the disk storage – When a computer has run out of RAM it is said to be “swapping” Some aligners have a minimum memory limit for optimal performance Human de novo requires lots of RAM! Disk space Unlike RAM, information is retained after the machine is switched off Speed of access is slower than RAM Magnetic disks have a seek time and read time Read data from a block is faster than seeking all over the disk to get the data Can be RAIDed to improve performance How much disk space? Space, or lack thereof, is the problem Understanding bandwidth Bandwidth of a connection represents the maximum rate of transfer between two points Most com