Day 3: Memory as first class citizen, Pointers, Virtual Memory, the Stack, Computer Architecture & Design

Casey-isms

• THINKING ONLY IN TERMS OF EFFICIENCY FOR THE ABSTRACTION LEADS TO SLOW, BUGGY CODE 
•  In general latency will be bad, but we can design programs to take advantage of throughput!
•  Programming always comes back to what is happening with CPU and memory.

Handy Dandy Visual Studio Tools

• Handy window for budgging Debug -> Memory
• You can look for a specific location in hex!

Memory

• CPU, GPU, audio, all take data from memory, rearrange, and replace into memory so the hardware component can read the information and execute correctly
• What is the CPU and memory actually?
• First, whether it be CPU or GPU (whatever you're programming)
• Take information into memory, modify, put out back to memory
• Al information takes, modifies and puts back out so other things can use that data & turn it into a correct signal

FUNDAMENTAL What is CPU doing to change memory?

• Basis of design of all code
• Basis of optimization
• Basis of all debugging
• In C/C++, memory is a first class citizen!

Pointers

• Boiling it down, it's a number that holds an address
• It boils down to this...
• Example: char unsigned Test;
• You're saying "I want what's in 8 bit memory" (WHAT)

• Example: char unsigned *TestPointer;
• I want to talk about the location of the 8 bit memory (WHERE)

• Example: TestPointer = &Test;
• & (address) I want to know where Test is in memory and assign to TestPointer

• This can be represented all on the same line using inline declaration
• char unsigned *TestPointer = &Test;

• char unsigned Test; (From watch)
• Value: 255
• char unsigned *TestPointer; (From watch)
• 0x003cf9af (number of bytes from bottom of memory aka first piece two where it stores Test. In decimal is 3996079
• TestPointer = &Test;

Small Overflow Example

• *TestPointer is pointing at the address of Test, therefore, they are the same and updating one will update the other.  
• They are functionally the same thing!!!

But what's the catch? VIRTUAL MEMORY

• In the old days, the memory address would be the actual location on the hardware. 
• Nowadays, that is not true because machines use virtual memory.
• Virtual memory allows processes to cooperate with each other.
• There are a lot of concerns like memory location, security, memory amount usage, etc.
• Allows running code what appears to be the memory of the machine, but not the real machine in terms of location.
• It does this using pages, which are chunks or ranges, that exist in physical memory.  
• But each process actually has its own concept of pages and thinks memory is laid out in a particular way.  
• Pages exists only when it accessed necessarily
• OS is in charge of shuttling physical memory to hard drive or rearranging layout using a table maintained on CPU that allows each processes view of pages to be dynamically mapped to where they actually are in physical memory
• One example is the TranslationLookAsideBuffer, Translation map table
• But when we code, we can think we're accessing physical memory
• But it's very important to keep virtual memory in mind so we can manage it properly.
• Pointers tend to be displayed in HEX due to its size.

Welcome to the Stack

• When did we ask the OS for memory?
• Who gives us the memory? Presenting...
• THE STACK, it a special location in memory given to us by the OS
• We don't explicitly ask for the stack! 
• System automatically asks the OS for it.
• This functionality is built into the compiler.  
• This is not true to super low level memory, where manual control is needed
• How does it work?
• Things go deeper, and then shallower....
• Grabs memory at the end of stack, and then grows
• Then at the end, it's removed
• Think of it as an audio bar. It's grows and shrinks
• The middle of the bar almost never disappears
• Since we're constantly calling, returning, calling, returning, calling, etc etc

Address space layout randomization

• Randomizes where your program runs in memory for security reasons
• Code, stack are in different locations
• This makes it much harder for hackers & more difficult to debug.
• We can turn it off to make it easier to debug
• Pages can be marked as readable, writable for security reasons
• This can be turned off and on
• CPU has flags that can mark makes as executable, but not writable

Basics of Computer Architecture & Design

• Speed of light / clock speed (i.e. 3.2 ghz) = 9.235234643 centimeters
• Which is approximately the distance from CPU to memory!
• In other words, going from CPU to memory, even at the speed of light is EXPENSIVE!
• Going out and getting things from memory is huge! 
• This is because of the limitations of wires and the actual location.
• Next-gen: Stack DRAM or RAM on CPU
• Similar problem with GPU/CPU/Memory triangle
• Latency- how long it takes one thing to start and finish something
• Throughput - how fast things can go through at peak congestion
• Bandwidth of memory related closely to Throughput
• In general latency will be bad, but we can design programs to take advantage of throughput!
• Because of latency, CPU's use L1, L2, and L3 cache right on chip to combat latency to minimize the time we have to talk to memory. It'll ask the cache, then memory (via the memory controller)