Systems Level Programming

2024-09-05

2024-09-05T12:00:00+00:00

2024-09-05

Logistics

What to expect in this class.

Discussion of policies and the landing page.

https://konstantinnovation.github.io

Resources

Most of the resources are listed in the summer assignment found on the systems page of my website. (this page)

Homework 01

1. Forms:

Please fill out the start of term survey

Please read and fill out the Computer Lab and Resources usage Policy

2. Working environment

Please ensure you have a working linux os on your home machine.

If you have a mac or linux computer, you already have one!

Mac

To work on a Mac computer you must install Xcode from the App Store. Run the following in a terminal in install xcode command line tools:

xcode-select --install

Windows

Windows users can install WSL (a linux virtual machine) on your windows computers:

https://docs.microsoft.com/en-us/windows/wsl/install

On all machines:

Verify the gcc command runs on your home machine.

go back to the top of the page

2024-09-06

2024-09-06T12:00:00+00:00

2024-09-06

Class overview

In this class you will be learning to write programs for a Unix-like system using the C programming language. You will learn about memory, processes, networking, and as a by product: how to better use a unix-like system (in this case Mint Linux).

Autonomy in my class

I treat you as if you were responsible individuals, even if you aren't. This means that I will not always check if you are doing what you are supposed to. You need to let me know if you are having any issues in class or at home.

If you do not understand even a single word, you need to speak up. In a technical class context clues are NOT sufficient to convey nuanced meaning that is needed. e.g. shibboleth

You also must read... all of the directions when trying to solve problems or work on assignments. Failure to do so will jeapordize your ability to succeed. A single word missed like "not" or "all" could mean you are solving the wrong problem.

prerequisite terminal skills:

Opening a terminal
Creating folders and files
File paths
Moving and manipulating files
Using SSH to run commands on other computers

Terminal Refresher tutorial:

If you feel you do not have these skills at this point, please refresh your memory here: https://ubuntu.com/tutorials/command-line-for-beginners#1-overview

Please note that only sections 1-5 are needed.

Homework 02

Post a greeting on piazza in the section for your period. You can additionally post if you have questions or issues with the summer assignment.
Get your ssh-keys working on your personal devices (see the landing page for my website for all classes for instructions).
Optinally : complete the Terminal Refresher tutorial sections 1-5

[back to top of page]

2024-09-09

2024-09-09T12:00:00+00:00

2024-09-09

C Quickstart

By convention, C source files should have a .c file extension (i.e. program.c).
The C compiler we will be using is gcc (the Gnu C Compiler)
usage: $ gcc program.c
- This will create a standalone executable file.
- The default name for the output file is a.out
- There is no preferred extension for c executable files (think about programs like ls, ssh, chmod, these are all C programs, notice the lack of file extension).
- You can provide your own output file name with the -o flag.
  - usage: $ gcc -o pro program.c
Compiled C programs are natively executable, to run them just type ./program (i.e. $ ./a.out or $ ./pro
C compilers assume you wanted to do what you did, and don't give errors for a variety of things you think should not work.
The ./ is only needed because you probably compiled the file in a directory(folder) outside your PATH environment variable (if that is unclear, you can forget the previous sentence entirely for now).
Before moving on, you should write, compile and run the example program provided below. I promise it is 100% sytactically correct. (Yes, you may get a compiler warning message, this is the only time you're allowed to ignore it.
```
int main(){
      printf("Hello world.");
```
Addendum: Return values

By default gcc will not tell you if a return statement is missing. Please use the -Wall flag to give you more warnings.

Additionally there is a special case in that the main function is to return 0 if control reaches the end without an explicit return. This only applies to the main function in C99, and older compilers will NOT guarantee this behavior. (This is not likely to be an issue for you.)

SSH Keys

Create SSH Keys on the lab computers if you haven't done so already.

Homework

Get your home computer setup to compile and run C programs

Test your home environment to ensure you can compile.

[back to top of page]

2024-09-10

2024-09-10T12:00:00+00:00

2024-09-10

printf

The thinking person's System.out.println

Extensive documentation can be found by using the command: man 3 printf

printf is the function normally used in C to print to standard out.
usage: printf( string, arg0, arg1, ...)
- Sends string to standard out.
- The first argument must be a literal string enclosed by ", as in the example program above.
- string can contain special placeholder characters that are used to insert other values into the output.
- %d is the placeholder to display a value as an int.
- %lf is the placeholder to display a value as a double.
- There are other placeholder characters that we will see later on.
- If placeholder characters are used, then they will be replaced by the arguments following the string when printf is executed.
- The value arguments can be either variables or literal values.
- example: printf("these are numbers: %d %lf\n", 3, 845.273); would display: these are numbers: 3 845.273
Once you get used to it, most people prefer printf's value replacement system to Java's string concatenation (+) inside System.out.println. In fact, Java does have a System.out.printf because of it.
The printf command does not add a new line at the end of the output, so you can use multiple printf statements to output on the same line.
As an exercise, add some printf statements to your existing example program. Try declaring variables and printing their values. Once you get the hang of it, try using the wrong formatting characters, see what happens…
Type Placeholder

int %d

long %ld

float %f ^*

double %lf^*

char %c

string %s

pointer %p
* %.xf or %.xlf will print up to x significant digits after the floating point
There are many formatting options, you should look at the man pages of printf for more options using man 3 printf

Type	Placeholder
`int`	`%d`
`long`	`%ld`
`float`	`%f` ^*
`double`	`%lf`^*
`char`	`%c`
string	`%s`
pointer	`%p`

[back to top of page]

2024-09-11

2024-09-11T12:00:00+00:00

2024-09-11

Piazza questions are really important today

Ask questions about todays notes on piazza. This was meant to have an in person lecture but I was not in school to give a demo + answer questions.

Binary

Internally to computers numbers are written in binary, which is base 2. The number 111₂ is actually the decimal number 7, but it is expressed in base 2.

Binary	Decimal
1₂	1₁₀
10₂	2₁₀
11₂	3₁₀
101₂	5₁₀
1000₂	8₁₀
1010₂	10₁₀

The way you convert from binary to decimal is the same as how you view the value of a base 10 number, you multiply the digit times the column value, e.g. 523 is 5*100 + 2*10 + 3*1. Since binary only has 0 and 1, you just sum the column values of each digit that is 1.

Incidentally a bit is a single binary digit. A byte is 8 bits.

Take this diagram, which explains how to determine the value of 11100111101₂

You don't have to be a master of binary, and I won't quiz you on converting, but you should understand the concept, and what a term like 32-bit integer or 4-byte integer means. They both mean a binary number with 32 digits (4 bytes = 4 x 8 bits).

You should know that the more digits a number has the more numbers you can represent, e.g. a 3 digit number in base 10 can be from 000 up to 999, so 1000 options. The more digits, the bigger the range of values, a 10 digit number has more possible values than a 3 digit number.

We designate the size of a variables use binary digits, or bits. The larger the number of bits (or bytes), the larger the range of values.

Primitive Types

All C primitives are numeric. The only differences are floating point vs. integer and size of variable in memory.
sizeof(type) can be used to find the size in bytes (stdlib.h).
Size of different types is platform dependent , you have to use sizeof to be certain.

Type	Size (bytes)	Range
`char`	1	-128 to 127
`short`	2	-32,768 to 32,767
`int`	4	-2³¹ to 2³¹-1
`long`	8	-2⁶³ to 2⁶³-1
`float`	4	7 digits of precision
`double`	8	14 digits of precision

char is an integer type, but can be used to refer to character literals as well.
- char c = 97; and char c = 'a'; are both equally valid statements.
- This also means you can perform arithmetic operations on chars natively.
Variables can be declared as unsigned. Unsigned variables do not use a bit to store the sign of the number, making the lower bound 0 and increasing the upper bound.

Type	Size (bytes)	Range
`unsigned char`	1	0 to 255
`unsigned short`	2	0 to 65,535
`unsigned int`	4	0 to 2³²-1

Note there is no boolean type. In c, any number is a boolean value:
- 0 is false
- All other numeric values are true
- the expression if(x = 2){} which has an assignment instead of a comparison does not cause an error. It evaluates the assignment and becomes: if(2){} This is desired behavior, but does NOT check if x has the same value as 2. x=0 is false while x = non-zero numbers is true. This is the source of bugs in many programs

Operator Precedence

Operators evaluate from left to right unless otherwise noted

() [] -> .
RIGHT TO LEFT ! ~ ++ -- +(unary) -(unary) *(de-reference) &(address of) (type cast) sizeof
* / %
+ -
<< >>
< <= > >=
== !=
& (bitwise and)
^ (bitwise xor)
| (bitwise or)
&&
||
RIGHT TO LEFT ?:
RIGHT TO LEFT = += -= *= /= %= ^= |= &= <<= >>=

Experiment!

Please experiment with some things.

Write the example and explanation in your notes.

Create some expressions using opeators and print the results. Many of the operators are the same as in java. Do not worry about dereferencing, address of, bit shifts (>>) and bitwise operators.
Make an example of using multiple assignment operators "=" evaluating from right to left.
How does the ?: (ternary operator) work, create an example.
The ternary operators in the same expression are right to left, what does this allow a programmer to do that would not work it it were left to right?

[back to top of page]

2024-09-12

2024-09-12T12:00:00+00:00

2024-09-12

Variables

C is statically typed, meaning every variable must be given a type.
Variables must be declared before they are used TYPE IDENTIFIER; e.g. int x;.
- You can assign a variable a value at declaration (i.e. int x = 10;)
THERE IS NO DEFAULT VALUE FOR VARIABLES
- In Java, instance variables got initialized to 0, arrays were also zeroed out. That’s a thing of the past.
- Remember that declaring a variable means requesting a piece of memory to be used by your program of the corresponding variable size. (int means you are asking for 4 bytes of memory.)
- If you do not initialize (provide a value for) a variable, its initial value will be whatever happens to be in the piece of memory that was assigned to your variable. Sometimes, that’s 0, sometimes it’s 2167354. You cannot predict this reliably.
- This is cause for one of the most frustrating kind of programming errors in C. Normally, if you run the same function with the same input twice, you will get the same result. If you don’t initialize a variable, you could run the same program twice and get two different results, because you’re not guranteed that variable will have had the same value twice (common occurance: you run your program and get lucky, a variable is initialized to 0. Then I run your work and the program crashes or does not give the required result because the variable is initialized to some junk value).
Variables can be declared as unsigned (i.e. unsigned int u;).
- unsigned variables have a lower bound of 0 and a higher upper bound than their signed counterparts.
- unsigned char uc; declares a 1 byte integer type that can hold any number in the range [0, 255] otherwise known as [0, 2^8 - 1], as in there are 8 bits used for the number.
- unsigned variables don’t need to set aside a bit for the sign of the value, hence the larger upper bound.
- There is an additional u in the printf placeholder string for unsigned types. "%d" becomes "%ud".

Functions

Function and variable names are both examples or identifiers.

All identifiers must be declared before they can be used.

A function declaration provides its return type, name and parameters.

This is also known as a function header: double foo(int bar)

A function body is almost always attached to the header to make a complete function:

double foo(int bar){
    return 1.23;
}

Loops and conditionals have the same syntax as java!

Since java and c have the same syntax for if/while/for you can experiement with them to make sure you can use them with real problems.

Lab01

Assignment01 repo

Select your name the first time you use our github classroom page. Please click on your name only.

Look at the problems here: http://projecteuler.net/index.php?section=problems

They are all math problems of some sort, you will be writing a program with several functions to solve them

Keep the file names provided in the repo.
Write a function for each problem you solve. You may wish to have auxiliary functions.
Each of your problem functions should return the numerical solution.
In your main(), call and print the results of each function in numerical order(problem number, not result) one per line as seen in the example below.
Please do the following: 1,5,6 after that try 14 and 2.

Sample output (not correct values):

  Euler01,1023
  Euler05,92381
  Euler15,31821.123
  etc.

[back to top of page]

2024-09-13

2024-09-13T12:00:00+00:00

2024-09-13

Compiling and Linking

Compilers are more complex than straightforward source code -> executable code translators, and have multiple components.

To start, we'll look at three major pieces of gcc, the preprocessor compiler and the linker.

Preprocessor

The preprocessor is, at it's simplest interpretation, a text replacement system.
Modifies source code file with text, as opposed to binary data.
All preprocessor commands start with # (i.e. #include <stdio.h>)
Note that preprocessor directives do not end in ;

A few basic preprocessor directives

#include
- Adds the entire text of the included file.
#define
- Usage: #define TEXT REPLACEMENT
- Will replace every instance of TEXT with the provided REPLACEMENT.
- Examples:
  - #define PI 3.14159
  - #define MESSAGE "Hello!"
  - printf("%s, %f\n", MESSAGE, PI); would turn into printf("%s, %f\n", "Hello!", 3.14159);.
- Note that define does not use =, becuase this is not an assignment.
- You can also use #define to declare function-like macros.
  - #define MAX(a, b) a > b ? a : b
  - MAX(x, y) would turn into x > y ? x : y
  - Note that this would be a direct copy/paste, so if x were a command that gives different results each time (random(), next(), x++, etc.), it would be pasted twice and likely evaluate to two different results.
#ifndef IDENTIFIER ... #endif
- Conditional preprocessor statement.
- If IDENTIFIER is not defined (for the preprocessor), then include all the lines of code up to the #endif.
- If IDENTIFIER is defined, skip everything up to the #endif.

Example of #ifndef

#ifndef PI
#define PI 3.14159
#endif

Compiler

Turns C source code into binary code.
The result is not an executable program.
Only one C file is compiled at a time.
The compiler checks called functions against their declared hearders, but if a function is defined in a separate file, its code is not added at this step.
$ gcc -c <FILE> will run the preprocessor and compile stages only, creating a non executable binary object file. The resulting file will have an extension of .o.
Since an executable is not created, you can successfully compile, via $ gcc -c, a C file that does not have a main function.

Linker

Combines compiled binary code from multiple files into a single executable program.
Will automatically look for standard library source code, or anything that can be included using <>.
If multiple definitions for any identifier is found, the linker will fail.
Must find one definition for main.
If you provide gcc multiple c files, it will compile each one individual and then run the linker on them together.
If you provide gcc any .o files, it will skip the compilation step for those files and then use them during linking.
You can mix and match .c and .o files for gcc, but it is not encouraged.
For example these would be good ways to compile a program, if you had files foo.c goo.c boo.c:
- $ gcc -o program foo.c goo.c boo.c
- or compile the files separately: (If you only changed one of the c files, you can recompile that one file but not the rest.)
```
$ gcc -c foo.c
$ gcc -c goo.c
$ gcc -c boo.c
$ gcc foo.o goo.o boo.o
            
```
  The output would be the default a.out but you could optionally use the -o flag to change it.

Homework:

Monday you will modify your lab, so make sure you have it working.

[back to top of page]

2024-09-16

2024-09-16T12:00:00+00:00

2024-09-16

Makefiles

Make is a command line tool to help automate building programs with multiple files and dependencies.
It is designed such that you can compile only the files that have been modified, or that rely on modified files.
Compiling instructions and file dependencies are put into a file called makefile or Makefile.
Running make, will look for a file with one of the default names but (you can specify a different file with the make -f FILENAME argument).
The main parts of makesfiles are:
1. Targets: Things to make (usually executables or .o files)
2. Dependencies: Files or other targets needed to create a target.
3. Rules: The steps to create the target.
Make will always run the first target if none is specified.
Make recursively goes through dependencies.
Make will check the modification timestamps for targets and dependencies and will only run the rules if the target is older than one or more of its dependencies.

Makefile Syntax: (<TAB> is the tab character)

target: dependency0 dependency1 dependency2 ...
<TAB>rule

There should be a newline between the dependency list and the rules, and the TAB is necessary, there should not be any space between it and the rule.

You can try to copy/paste this makefile, but replace the leading spaces (margine only) with tabs.

default:
    echo "'make' will run this line"
compile:
    @echo "'make compile' will run this line, but the @ suppresses the echo command on the terminal!"
run:
    @echo "'make run' will run this line, but the @ suppresses the echo command on the terminal!"

Here is a makefile for a program made from three .c files: main.c, foo.c and goo.c.
main.c calls functions from foo.c

foo.c calls functions from goo.c

Note: copy/paste from a website does not preserve tab characters! DO NOT COPY PASTE, write it yourself with tabs

all: main.o foo.o goo.o
  gcc -o program main.o foo.o goo.o

main.o: main.c foo.h
  gcc -c main.c

foo.o: foo.c goo.h
  gcc -c foo.c

goo.o: goo.c goo.h
  gcc -c goo.c

This makefile creates the executable file program.
Since all is not a file and it is the first target, it will always run.
- If instead, the first target was called program, then make would check the modification timestamp of that file.
main.o is the first dependency, so make will go to that target.
main.o depends on main.c and foo.h
The rest of the dependencies will go through in a similar way. Running $ make the first time would do the following:
```
gcc -c main.c
gcc -c foo.c
gcc -c goo.c
gcc -o porgram main.o foo.o goo.o
      
```
Notice the order of compilation and trace it through the makefile.

If goo.h is modified, the following would happen:

gcc -c foo.c
gcc -c goo.c
gcc -o porgram main.o foo.o goo.o

Actual Example:

main.c

#include <stdio.h>
#include "foo.h"
int main(){
  printf("fu(5) is: %d\n",fu(5));
}

foo.h

Any .h file only contains headers and an include guard:

#ifndef FOO_H //include guard
#define FOO_H
int fu(int x);
//more function headers can go here
#endif

foo.c

int fu(int x){
  return x+1;
}

makefile

This file should be named makefile or Makefile

Please Note: the clean target will remove any compiled files and executables, so that you can test your compilation from scratch. This is very important.

Also Note the compile target also has a target "runme" so that it is called EITHER when you use make compile, OR when the file runme needs to be created.

The first target is the default, no matter what you name it.

notDefault:
  @echo "No arguments runs the first recipe no matter what you call it."
compile runme:  main.o foo.o
  @gcc -o runme foo.o main.o
main.o: main.c foo.h
  @gcc -c main.c
foo.o: foo.c
  @gcc -c foo.c
run: runme
  @./runme
clean:
  rm *.o
  rm runme

Update Lab01

Big goal: Set up a makefile for your lab01 and link it as described below

I will ONLY compile and run using make. You will lose credit for having invalid makfiles.

Modify your euler assignment such that you have the following files:

euler.c
- Your euler assignment, modified to only contain the euler problem functions.
- Get rid of main() by moving it to main.c.
- Do not put separate functions headers here (function definitions will have the header line still of course).
euler.h
- The function headers for the functions in euler.c
- They should be no argument functions that return a single int. This includes if the intermediary results exceed integer range.
- Name them using the format: int euler01(), int euler01(), etc. Just the problem numbers you were assigned.
main.c
- Add #include "euler.h" at the top.
- Has a main function that correctly runs all the functions in euler.c
makefile
- Appropriately compile euler.c and main.c separately.
- Generates an executable file called euler
- The compile target should be to compile the entire program.
- The run target will run your program. Both make run, and ./euler should run your program.
- I will ONLY test working makefiles. Test with make compile and make run
- For my labs, you MUST use the @ operator before each command of your makefile to suppress the command text, and only see the output.
- If running make run on a clean clone of your lab01 repo works on the lab machines, then you are good to go. You should make a clean target to test this on your non-clean repo that has the extra files.
  output format after using make run:
```
Euler01,1023
Euler02,55552
Euler05,92381
Euler15,31821.123
etc.
```

[back to top of page]

2024-09-17

2024-09-17T12:00:00+00:00

2024-09-17

Memory

Bits & Bytes

All digital data is binary, broken up into series of 1s and 0s.
Physically, this data can take many forms, electronic (high voltage | low voltage), optical (light on | light off), magnetic (+ | - magnetic charge) and so on.
A single 1 or 0 is a bit, a unit of digital data.
8 bits make a byte.
- Why 8? - Some people thought that was a good number. It used to be 4.
If you have a music file that is 5 MB (megabytes) large, that means it takes 5 million bytes, or 40 million bits to be represented digitally. That's 40 million individual 1s and 0s in whatever physical form it may be.

Computer Memory 101

In order to get all this pointer stuff down, we have to be cool with understanding computer memory. So let's do that.
In general, memory is the term used to describe the computer part that contains any active data. Active data includes things like:
- All open applications, even ones running in the background.
- All open files.
- Operating system.
- Background processes.
Let us say a computer has 4 gigabytes of memory, that means it can handle 4 billion bytes of data open at once.
The important distinction to be made is between memory and storage.
- Storage refers to data stored on disk (Hard Drive, SSD, Flash Drive, Floppy Disk…).
- Storage is where data gets saved for the long term. At any given time, most computers will have a lot more data in storage than in memory.
- For example, you might have the entire Pearl Jam discography stored on your hard drive, but you can only listen to one song at a time, so the song you-re currenlty listening to would be the only data in memory out of all the other songs in storage.
Memory is much faster to access than storage, which is why it is used. THe downside is that memory is volitile, meaning it requires power to retain data (imagine losing everything when you shut your compuer off), and it's much more expensive than storage.
The most common form of memory is RAM (Random Access Memory). The more RAM you have, the more data you can hape open at once.
- Computers can use a concept called virtual memory, which will allocate unused disk space for memory purposes in the event that your RAM is full.
When you open a program/file (in the *nix world, everthing is a file), the file is copied from storage into memory. Saving a file reverses this process, taking the changes you-ve made from memory and copying them into storage.

Interacting with Memory in your Programs

Just like any other program, when a program you write is run, it takes up memory space.
Every variable and function you write gets turned into bits which takes up memory space when run.
In this class, when we talk about the memory usage of a program, we will mostly be talking about variables.
For example, when you declare an int, that means your program will request 4 bytes of memory to store a value. See prior notes for the chart of types and sizes.
Some more detail on endianness:
- Forgetting about computer data for a moment, think aboput normal decimal numbers. In the number 2,354 we would say 2 is the most significant digit, because that 2 represents 2 thousand, the largest value of any digit in that number.
- Generally, we write decimal numbers left->right from most->least significant.
- Endianness is a similar concept, except instead of thinking of the significance of digits, we look at the significance of bytes.
  1. Consider the value 261. In binary, that would be: 100000101, which is a 9 bit number.
  2. To store 261 in an int, C will use 4 bytes, so it would really look more like this:
    - 00000000 00000000 00000001 00000101
  3. Think about the significance of bytes in the same way you think about the significance of digits. In the above representation, the most significant byte comes first. Since we only need 9 bits (which is spread over 2 bytes) to represent 261, the first two bytes are all 0. The third byte, 00000001, represents the number 256.
  4. Systems that use this representation are called big endian.
  5. Other systems use the reverse order, going from least significant to most significant byte. These are called little endian.
  6. 261 in little endian format would be:
    - 00000101 00000001 00000000 00000000
    - Notice that the indicidual bytes are in most->least significant bit order, but the order of the bytes is reversed.
  7. Another example, 2,151,686,160
    - Big endian: 10000000 01000000 00100000 00010000
    - Little endian: 00010000 00100000 01000000 10000000

[back to top of page]

2024-09-17

2024-09-17T12:00:00+00:00

2024-09-17

Memory Addresses

In order to understand pointers, we need to take a closer look at what a variable is

There are three key features to any variable in your code:

The identifier: Name you use in code to refer to the variable.
The value: Data that you store.
The address: The location of data in memory.

Let's focus on the address (the following explanation is somewhat simplified, the nitty gritty details are not necessary to understand the concept)

Memory is addressed by starting at the first byte block (1), going up until the last accessible byte.

For example, if your computer has 4GB of RAM, then in theory memory addresses would go from 1 to ~4,000,000,000.

The amount of potential memory addresses is limited by the processor, since a processor must be able to read an entire memory address within a single cycle. For modern, 64-bit computers, this means you could theorhetically have 2^64 bytes of memory, though this number is practically limited by hardware.

The address space of a program is determined by the operating system (OS), when the program is run. Therefore it can be different each time.

Get an address of a variable:

You can get the address of any variable using the address of operator : &;

The %p placeholder character will print out a memory address in hexadecimal format: printf("x: %d, address of x: %p\n", x, &x);.

If you would prefer to see the address in decimal, you can use the placeholder for an unsigned long on a 64 bit. (This will most likely result in a warning from gcc): printf("x: %d, address of x: %lu\n", x, &x);.

Store an address

In order to set a pointer to an existing variable you use the & operator

int x = 97;
int* ptr;//declar a pointer to an int type.
ptr = &x;//set the value of ptr to the address of x (make ptr point to x)

Get the value at an address:

In order to follow a pointer to where it points, you must de-reference it.

Dereference by using the * operator. This means ptr is the address of x, but *ptr is the contents of x which happens to be 97 in this example.

Modify an address:

You can modify what address the pointer ptr contains by assigning as follows: ptr = NEWER_ADDRESS; or equivalent expressions.

Modify the contents of the address:

You can modify the value contained in the memory slot ptr points to using the assignment: *ptr = NEWER_VALUE; or equivalent expressions.

Lab02

Do the following in a C program.

Clone the repo from here to do your work:

https://classroom.github.com/a/qB71UL4A

It will get a little tricky, so do things one step at a time, compile, run and test frequently.

Declare and initialize variables of types char, int, and long.
Print out the addresses of each variable in hex and decimal.
- What do you notice anything about the addresses?
- What about when you run the program more than once?
Declare and initialize pointers for each of your variables.
Print out the value of what each of the pointers are referencing. (this should be the same value from step 1.).
Use the pointers to modify the values of the original variables and print out the new values.
Declare and initialize an unsigned int variable and 2 pointers that point to it, one should be an int* and the other should be a char *
Print out the value of each pointer (this should be the memory address), and de-reference each pointer to print out the value each points to.
- The output should look something like: p: 0x7ffee3dbd938 p points to: 133.
Print out your unsigned int in decimal and hex.
- The printf formatting character for a hexadecimal int is: %x
- The printf formatting character for an unsigned int is: %u
Use the char* to print out each individual byte of your unsigned int.
- The printf formatting character for a single byte in hex is %hhu for unsigned decimal integer and hhx for an unsigned hex, (that is half of half of an integer).
Super helpful option: Try setting the value of your unsigned int to: 1 + 2 + 4 + 256 + 512 + 1024 and then print out the bytes again.
Use the char* to increment each byte of the unsigned int by 1. Print out the unsigned int in both hex and decimal after each modification. When done, print out each byte like in step 9. You may need to reset the char * so that it points to the unsigned int, depending on how you wrote step 4 out.
Perform the same operation as in step 11, except add 16 to each byte.

As you go through this, look at what's happening to each byte, and think about the level of control you have over the memory space used by your program. What values did you see for each individual byte? Can you make sense of them? Does the order make sense?
If you want to test things further try doing all the above with different kinds of values:
Try an unsigned int with a value greater that 2.1 billion.
Try a regular int.

[back to top of page]

2024-09-18

2024-09-18T12:00:00+00:00

2024-09-18

SSH

The documentation for SSH on my site has been updated to incldue multiple ssh targets in case one is non-functional ( We are looking at you marge... )

https://konstantinnovation.github.io/index.html#ssh

the -fu suffix:

-fu : (slang) Used to form nouns indicating expertise or mastery of specified skill or area of knowledge.

Google + ‎-fu → ‎Google-fu ‎
script + ‎-fu → ‎script-fu
My Google-fu is weak!
Aragorn uses Ranger-fu to figure out that Sam and Frodo have taken a boat.

Etymology

From kung-fu.

More `printf`

printf provides different ways of printing out values.
Remember that no matter how you write a number in your code, it is stored in memory in binary, so as far as printf is concerned, even printing out a value in decimal (base 10), requires translating the stored data.
There is no such thing as a “native” hexidecimal or decimal number. You write them in your code in some way, and they turn into binary when the program is compiled.
So printf takes the binary data, and displays it in a particular way based on the formatting character(s) provided.
%d : print a value as a signed decimal int.
%u : print a value as a decimal unsigned int.
%o : print a value as an octal number.
%x : print a value as a hexidecimal number.
- %o and %x will always treat the value as if it were unsigned.
- You can print a value out with %u or %d regardless of how it is declared, that doesn-t mean it will make sense, just that printf will convert the value accordingly.
h : modify the printed value to look at 2 bytes instead of 4.
hh : modify the printed value to look at 1 byte.
h and hh can modify u, d, o or x.

The code snippet below displays these options: (note %% prints a % symbol.)

unsigned int q = 2151686160;
printf("%%d: %d\n", q);
printf("%%u: %u\n", q);
printf("%%o: %o\n", q);
printf("%%x: %x\n", q);
printf("%%hhx: %hhx\n", q);
printf("%%hhu: %hhu\n", q);

Note: 2151686160 is 80402010 in hex, and 1000 0000 0100 0000 0010 0000 0001 0000 in binary.

When run, this prints (on my computer):

%d: -2143281136
%u: 2151686160
%o: 20020020020
%x: 80402010
%hhx: 10
%hhu: 16

Things to notice:
- %d is not the actual value we used, this has to do with how negative numbers are represented.
- %hhx and %hhu print the first byte of the value. Based on that, you can tell the endianness of the system the program is run on (it is possible you get a different result from my example).

Bash Scripts

Quick discussion on the reasons to use a bash script vs make.

Make

Make is specialized in compiling source code and building projects
bash scripts are a general purpose tool
The ability to have dependencies compiled automatically
only compile files that are older than their depenencies

Bash scripts

Bash scripts are just files that run a predetermined set of terminal commands.

Create a file myScript.sh

Add the following code:

#!/usr/bin/bash
S="Ding"
for i in {1..9}
do
	S=$S" ding"
done
cowsay -f fox $S

Chmod the file to be executable: chmod +x myScript.sh
Run the script using ./myScript.sh

The first line of the script says to use bash and not zsh to run your script. This is optional, but suggested if you use loops which differ in bash/zsh

Classwork + Homework

Write a bash script that is intended to be executed in the main directory of your repo. The script (minimally) does the following:

Get the link remote repo from the current repo (use git remote -v)

Here is a command to help you put it into a variable: cloneurl=`git remote -v | cut -d ' ' -f 1 | cut -d $'\t' -f 2 | head -n 1`
Clone that repo in the parent directory of the already cloned repo. Specify a name that is not going to conflict with your other repositores using: git clone GIT_SSH_LINK NAME_OF_FOLDER .
Change into that newly cloned repo.
Compile and run the code using
```
make compile && make run
```
You can visually determine if the output is as expected.
Remove the freshly cloned repo.
After that works try to use a bash if statement to look at the returned value of make compile You should run the program if it compiles, or print out "failed to compile" when it doesn't compile.
Extra: You can try to do more interesting things by looking at stdout &gt and stderr 2&gt terminal redirectionand and verifying that there are no errors automatically! I am not providing notes on this but google a few things and see if you can hack together a bash script!

You are permitted to use your google-fu, and share ideas/resources with others!

Note a bash conditional has the following syntax:

if command ; then
    echo "Command succeeded"
else
    echo "Command failed"
fi

[back to top of page]

2024-09-19

2024-09-19T12:00:00+00:00

2024-09-19

Lab02 deadline

The lab is due Monday. You will have the majority of the period to work on it today.

Math

The math library in math.h has a bunch of useful functions such as:

double sqrt(double x) : Returns the square root of x.
double log10(double x) : Returns the common logarithm (base-10 logarithm) of x.
double pow(double x, double y) : Returns x raised to the power of y.

The only problem is that when you #include &ltmath.h> you must compile with the -lm flag. (Both compile and linking steps)

Makefile tweaks

Don't use make run as a command like this:

    default:
      make run

Instead here is a better way to have make run be default, but easily change it later. It also gives you an idea of how to suppress rm having no target files.

Note the -lm flag on gcc when compiling files that use the math.h library.

.PHONY: clean run compile
run: prog
  @./prog
clean:
  rm -f *.o prog
compile prog: tester.o fun.o
  @gcc -o prog tester.o fun.o -lm
tester.o: tester.c fun.h
  @gcc -c tester.c
fun.o: fun.c
  @gcc -c fun.c -lm

Notice the following:

In order for the run command to automatically compile: The output filename prog is also a target. (next to compile) This allows the compiled file to be a dependency.
The .PHONY targets are NOT files. This is important if you have
The clean command is not @ suppressed. This is my preference so I see the rm command on the terminal. All other things should be suppressed unless you are debugging.

[back to top of page]

2024-09-23

2024-09-23T12:00:00+00:00

2024-09-23

Why make a script

What are some of the advantages of making a script to complete a task?

Where sholud scripts that you write be stored?

A quick bash scripting reference can be found here: https://devhints.io/bash

Control your environment

Everyone should be comfortable tweaking and configuring their working environment on a computer. Today you will learn to make your scripts more accessible when you are working in the terminal, so that you don't have to remember where you place them in order to call them.

The tool you created yesterday is a perfect example of something you want to use in multiple places (each of your lab folders). It does not make sense to copy this script to each lab folder however!

You can just evoke the script by typing /PATH/TO/SCRIPT/ScriptName.sh e.g. ~/scripts/ScriptName.sh

Here is a super simple version of the script:

#!/usr/bin/bash
cloneurl=`git remote -v | cut -d ' ' -f 1 | cut -d $'\t' -f 2 | head -n 1`
cd ..
git clone $cloneurl temprepo
cd temprepo
make compile
make run
cd ..
rm -rf temprepo

Note: #!/usr/bin/bash is required if your shell is not bash and you want to use the script by name. Otherwise you need to type bash scriptname.sh

If this fails to find bash, type which bash to get the correct location.

Updating your path variable

You may have noticed: you cannot just type the name of your script to trigger the script.

All programs and scripts you make require a ./ before the script/program name because they are not located in any of your PATH directories.

In the lab, you cannot put your programs in a pre-designated PATH directory, so you can just set a local directory to hold all of your programs and scripts.

Make a direcotry for your binaries/scripts

Make a directory ~/bin
```
mkdir ~/bin
```
Now you can make this directory part of your path using the command:
```
export PATH=$PATH:~/bin
```
Suggested: add this command to your ~/.profile so that you don't have to type it every time you use the computer.
Place your script clonetest.sh (or whatever you want to call it) from yesterday into your ~/bin directory.
Now the moment you have been waiting for: you can now use your script from ANY directory just by typing clonetest.sh, AND you can auto complete the command name with the tab key!

Any scripts and programs you place in this folder can be called by name much like ls, pwd and all those useful commands you use already.

Not sure which profile is executing?

It is possible you are editing the wrong profile since there are several. Between .profile, .bashrc, .bash_profile, or even .zshrc it can be confusing!

If you ever need to check what is running, just place an echo command inside your profiles so you can test which is actually being executed.

Tweak your script from yesterday

The git clone command can be a large wall of text you don't want to see.

You can silence this command by redirecting the stderr (error text) to /dev/null.

Try this to make the clone quiet: git clone URL NAME 2&gt /dev/null

Open a remote repo in a browser using the CLI including scripts:

Like this: firefox "https://`(git config remote.origin.url | cut -f2 -d@ | tr ':' /)`"

Change the explorer.exe to some other application... like opera on mac, you can use the open command , or explorer.exe on windows

Make it a shortcut:

alias gh='explorer.exe https://$(git config remote.origin.url | cut -f2 -d@ | tr ':' /)'

WSL

Demo utility of WSL.

Homework

There is a perusall assignment on academic dishonesty. Please Read it and ask any questions about it by tomorrow. Do not worry about the auto scoring.
Use your google-fu to learn some terminal-fu. (Specifically about the bash shell or scripts).
Make a post on piazza in the terminal-fu section. Either share something useful/cool/surprising about the shell, or add on to someone else's post with related cool stuff.

[back to top of page]

2024-09-24

2024-09-24T12:00:00+00:00

2024-09-24

Goals

Clarify issues with poiners and pointer arithmetic.
Talk about wsl features.

Pointers

Regular variables are designed to store values.
Pointers are variables designed to store memory addresses.
Pointers are variables, meaning they are an identifier for a value stored in memory at a particular address (see above for detail), the only difference is that a pointer is designed to store an address.
Pointers must be able to store the value of any potential memory address. On 64 bit computers, this means pointers have to be able to represent 64 bits, or 8 bytes.
Pointers are designed for addresses, which means they are natively unsigned.
Even though all pointers are the same size, we declare them using the type of the value pointed to.

Syntax

* is used to declare a pointer variable.
```
int x = 5;
int *p = &x;
```
Here p is a pointer variable that stores the address of the variable x.
Notice that p is a normal variable, and has its own, different, memory address.
If you-re thinking, “hey, this looks familiar… like object variables in java”. You-re right! Object variables, or references, are java's pointers. You just don-t have as much control over them as we do in C. In fact, think about the error you get when you try to use an uninitialized object variable in java… null pointer, meaning the reference stored is 0 (null), which is an invlaid memory address.
* is also used as the de-reference operator. This will return the value stored at the memory address pointed to by the pointer.
Given the definitions of x and p above:
int y = *p + 10; would set y to the value 15.
*p = y; would set the value at the memory address stored in p, to whatever the value stored in y is.

Pointer Arithmetic

Consider the following C snippet:

  unsigned int i = 2151686160;
  int *ip = &i;
  char *cp = &i;

ip and cp will store the address of the first byte used to store i. Depending on the endianness of the system, that byte will either be 10000000 (big) or 00010000 (little).
Let's just say that the first byte is located at memory address 3000 (using small number for ease of discussion)
Due to pointer arithmetic: If you perform ip++ and cp++, each pointer will be incremented by 1 * sizeof(type) , ip will increase to 3004 and cp will increase to 3001. In essence, ip would move one int forward in memory, while cp only moves one char (byte) forward.

Looking ahead

Using pointer arithmetic is needed to make use of arrays. An array[0] points to the start of a block of memory, while array[1] points 1 * sizeof(type) from the start. This is the elegance of arrays in c.

Endianness

Some more detail on endianness
Forgetting about computer data for a moment, think aboput normal decimal numbers. In the number 2,354 we would say 2 is the most significant digit, because that 2 represents 2 thousand, the largest value of any digit in that number.
Generally, we write decimal numbers left->right from most->least significant.
Endianness is a similar concept, except instead of thinking of the significance of digits, we look at the significance of bytes.
- Consider the value 261. In binary, that would be: 100000101, which is a 9 bit number.
- To store 261 in an int, C will use 4 bytes, so it would really look more like this:
  - 00000000 00000000 00000001 00000101
- Think about the significance of bytes in the same way you think about the significance of digits. In the above representation, the most significant byte comes first. Since we only need 9 bits (which is spread over 2 bytes) to represent 261, the first two bytes are all 0. The third byte, 00000001, represents the number 256.
- Systems that use this representation are called big endian.
- Other systems use the reverse order, going from least significant to most significant byte. These are called little endian.
- 261 in little endian format would be:
  - 00000101 00000001 00000000 00000000
  - Notice that the indicidual bytes are in most->least significant bit order, but the order of the bytes is reversed.
- Another example, 2,151,686,160
  - Big endian: 10000000 01000000 00100000 00010000
  - Little endian: 00010000 00100000 01000000 10000000

WSL demo.

You can open your current directory in windows explorer by typing the command explorer.exe .
Your wsl can see your windows drive in /mnt/c/ , and your windows install can see your wsl drive.
You can open windows programs on your linux filesystem as well! try it with a program installed like: /mnt/c/Program\ Files/Notepad++/notepad++.exe
Alias any windows programs in your config file with something like: alias notepad="/mnt/c/Program\ Files/Notepad++/notepad++.exe"
You can make a bash script to run pulsar or notepad++ really easily. Create a file called npp in your ~/bin directory of wsl. Then add the script:
```
#!/bin/bash
/mnt/c/Program\ Files/Notepad++/notepad++.exe $1
```
The $1 means the 1st argument of the script. If you then typed npp hello.txt it would replace $1 with hello.txt

[back to top of page]

2024-09-25

2024-09-25T12:00:00+00:00

2024-09-25

Random Numbers

Note: On some systems, you may be able to use other functions than the ones described below, but they are not standard for linux, so you should stay away from them for now.
Generating a random number in C requires 2 steps
1. Seeding the random number generator
- srand( time(NULL) );
  - srand(<SEED>) seeds the random number generator with the provided argument.
  - If you use the same argument to srand() multiple times, you will get the exact same sequence of random numbers.
  - time(NULL) will return the current EPOCH time, it is commonly used with srand() to get new random sequences.
    1. Getting a random number
- int x = rand();
- Returns the next random number in the sequence seeded by srand().
- Returns an int.
srand() and rand() are both in <stdlib.h>
time() is in <time.h>

Here is a script to repeat your make command 30 times to test random:

 for i in {1..30}; do make; done

Here is a 'random' script:

S="Ding"
for i in {1..9}
do
        S=$S" ding"
done
cowsay -f fox $S

Arrays

An array is an allocated block of memory meant to hold multiple pieces of data of the same type.
C arrays do not have a length attribute/function.
We will use [] to access array elements.
The size of an array must be set at declaration and cannot be changed.
The size of an array cannot be dyanmic.
There is no boundry checking (much more on this later).
Array declaraion/access syntax:
```
float ray[5];
ray[2] = 8.22;
```
The above code requests a block of memory large enough for 5 floats (20 bytes), which then can be accessed using 0-based [] notation.

Array Variables

Array varibles (not the arrays themselves) are pointers to the allocated array block.
Unlike standard pointers, array variables are immutable, meaning that you can never change the memory address an array variable points to.
In the previous example, ray is a variable that points to the beginning of the 20 bytes allocated to that array of floats.
The sizeof function can be used to find the size of a given type (like float or char *), or _the amount of memory associated with a given variable.
- sizeof(ray) would return 20.
- sizeof(ray) / sizeof(float) would return 5. It is more standard in C not to use this, instead using other constants/variables to keep track of array sizes. Since array sizes must be set at compile time, you-re more likely to see something like this:
```
int ARR_SIZE = 10;
double trouble[ ARR_SIZE ];
```

Array Variables & Pointers

Since array variables are pointers, we can assign normal pointers to array variables.
```
float ray[5];
float *rp = ray;
  
```
ray, is immutable, so we could not do soemthing like ray++, but rp is a normal pointer, so we could do rp++. Due to pointer arithmetic, rp++ would actually add 4 to rp.
sizeof(ray) would return 20, while sizeof(rp) would return 8, since rp is a pointer and only holds an 8 byte (on most systems) memory address.
This is commonly done, and because of pointer arithmetic, you can iterate through an array by using a pointer and incrementing it.

Array Indexing and `[]` Notation

The following two pieces of code perform the same task: ray[3] and *(rp + 3)
In the second example, we add 3 to rp, which is the same address as the location for ray[3].
The de-reference operator (*), is then used to retrieve the value.
You can think of the standard [] notation in terms of specifying an offset from the beginning memory address of an array.
Arrays are 0-indexed because the first element is stored at the starting address, so you need not add to get to the correct memory address.
The a[i] notation is actually shorthand for:
- *(a + i),
This means that you can use [] with pointer variables as well.
- rp[3] is valid code.
You can write ray[-1] or rp[-1], which would go to the value 4 bytes (one float size) before the beginning of your array.
If you use an index past the end of an array allocation, you will be attempting to access the memory addresses past the end of the array.
In either case, going past an array allocation on either end is not advised. Your code will compile, but when run, at best you-ll access other variables within the program, at worst, you-ll crash.

WARNING: HORRIBLE SYNTAX AHEAD

Once again, *(a + i) is the same as a[i]
+ is a commutative operations, meaning a + i == i + a
… some of you may see where this is going
Since: *(a + i) == *(i + a)
It follows that: a[i] == i[a]
So ray[2] can also be written as 2[ray]. Try it once, then NEVER DO IT AGAIN!
Use this knowledge wisely.

Lab03 Arrays

Due: Monday 09/30 8:00am

You will have today + 1 full class day + home time for this.

You SHOULD be done with this by Friday.

Clone link: https://classroom.github.com/a/9lGpCz-A

Do the following in a C program.

For this assignment, do not create helper functions, put everything inside main (it won't be too long). We will talk about passing arrays and pointers are function arguments next week.

Create an array large enough to store 10 ints.
Set the first value in the array to 0.
Populate the rest of the array with random values in the range -128 to 127.(you can make a function to generate )
Print out the values in this array in the format "[1, -3, 4, 7....,9]" that is: ", " between values with [] around the whole set.
Create a second array of the same size.
Create pointer variables that will point to each array.
USING ONLY POINTER VARIABLES (that is, do not use the array variables) do the following:
- Populate the second array with the values in the first but in reverse order
- Print out the values in the second array using pointer notation *
- Do this last step again using array notation (try the array variable and the pointer variable with the [] ).

go back to the top of the page

2024-09-27

2024-09-27T12:00:00+00:00

2024-09-27

Remember your #ifndef

In your header files remember to use the ifndef operator to not re-include things. This won't break now, but it will later and you won't know why.

A good naming convention for filename.h is to use filename_h as your ifndef identifier

#ifndef filename_h
#define filename_h
int foo();
void bar(int x);
#endif

C Functions

pass by value

All functions in C are pass by value.

This means that the arguments are copied into new variables when the function is called.

As a result, normal values are not modified when passed into a funtion. Consider this function to swap two values:

void swap(int x, int y) {
  int temp = x;
  x = y;
  y = temp;
}
int main(){
  int a = 10;
  int b = 20;
  swap(a, b);
  //what happened to a and b?
}

In this example, when swap is run on x and y, a and b are created. The function swaps the values of a and b, but once the function finishes, it is popped off the call stack, and a and b are gone. x and y are left unchanged.

The Power of Pointers!

This is where pointers come in handy. If you pass a pointer to a memory address, then you can modify the value it points to. Look at this modified version of swap.

void swap(int *x, int *y) {
  int temp = *x;
  *x = *y;
  *y = temp;
}

int main(){
  int a = 10;
  int b = 20;
  swap(&a, &b);
}

Now that swap takes pointers, we can de-reference the parameters to get at the values pointed to. When the function is colled, a and b become copies of the addresses of x and y, so when swap finishes, the values will actaully be swapped.

Pointer paramters

Passing pointers as arguments is actually what happens in java when object variables are used! You may recall the phrase pass by reference, all that means is pass by value, but the value being passed is a memory address.

When you pass an array as a argument to a function, the entire array is not copied. Since array variables are pointers, all arrays are treated as regular pointers when passed into a function.

The following function headers are equivalent:

void arr_func( int arr[]);
void point_func( int *arr);
It is generally preferred to use the second option, since it makes clear that the parameter is a normal pointer. It is possible to use the first option and think that something special is going on due to the array notation (but nothing is).
The sizeof() function will always evaluate to the sizeof a pointer in both cases (8 on 64-bit systems) and not the array size, as was the case when used on the immutable pointer variable you created originally.(see demo below)

Array parameter Example:

 #include &ltstdlib.h>
#include &ltstdio.h>

//n should be int*
void passArray(int n[]){
  printf("  size of n: %lu\n",sizeof(n));
  printf("&n %p\n",&n);
  printf("n %p\n",n);
}
int main(){
  int ARR_SIZE = 10;
  int nums[ ARR_SIZE ];
  nums[0]= 0;
  printf("size of nums: %lu\n",sizeof(nums));
  printf("&nums %p\n",&nums);
  printf("nums %p\n",nums);
  passArray(nums);
  return 0;
}

Warning:

array.c: In function ‘passArray-:
array.c:4:35: warning: ‘sizeof- on array function parameter ‘n-
          will return size of ‘int *- [-Wsizeof-array-argument]
4 |   printf("size of n: %lu\n",sizeof(n));
  |

Output:

size of nums: 40
&nums 0x7ffda2ebcf30
nums 0x7ffda2ebcf30
size of n: 8
&n 0x7ffda2ebcf18
n 0x7ffda2ebcf30

[back to top of page]

2024-09-30

2024-09-30T12:00:00+00:00

2024-09-30

Stack and Heap Memory

Every program has memory for both the stack and heap.

Stack memory

Stores all normally declared variables (including pointers and structs), arrays and function calls.
Functions are pushed onto the stack in the order they are called, and popped off when completed.
When a function is popped off the stack, the stack memory associated with it is released.

Heap memory

Stores dynamically allocated memory.
Data will remain in the heap until it is manually released. (or the program terminates)

Dynamic memory allocation

void* malloc(size_t x)
1. Allocates x bytes of heap memory.
2. Returns the address at the beginning of the allocation
3. Returns a void * so you should typecast:
  (type*) malloc(number-of-bytes);
  e.g.
```
int *ptr;
ptr = (int*) malloc(100 * sizeof(int));
```
free(void * p)
1. Releases dynamically allocated memory.
2. Has one parameter, a pointer to the beginning of a dynamically allocated block of memory.
3. Every call to malloc/calloc should have a corresponding call to free.
void* calloc(size_t n, size_t x)
1. Allocates n * x bytes of memory, ensuring every bit is 0.
2. Works like malloc in all other ways, including your need to typecast to the pointer type from void*
```
int *p;
p = (int*)calloc( 5, sizeof(int) );
```
void* realloc(void *p, size_t x)
1. Changes the amount of memory allocated for a block to x bytes.
2. p must point to the beginning of a block.
3. Returns a pointer to the beginning of the block (this is not always the same as p)
4. If x is smaller than the original size of the allocation, the extra bytes will be released.
5. If x is larger than the original size then either:
  - If there is enough space at the end of the original allocation, the original allocation will be updated.
  - If there is not enough space, a new allocation will be created, containing all the original values. The original allocation will be freed.

Visualizer is possibly helpful for the differences:

C Visualizer with array example.

[back to top of page]

2024-10-02

2024-10-02T12:00:00+00:00

2024-10-02

Checking for prime numbers

A naive way to check if an integer k is prime, is to check divisibility of all integers [2,k-1].

Slightly less naive is to check divisibility of all integers [2,x], and figure out how small an x you need.

Since all factors are paired around the sqrt, you can use the range [2,(int)sqrt(k)]

This is very slow as we don't need to check divisibility by all of those numbers. It would be faster to just check the prime ones.

Sieve of Eratosthenes

Here is a much better method to find prime numbers.

Assume all numbers are prime except 0 and 1.
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Start with the first prime number (2) Iterate through all prime numbers and cross them out because they are composite (not prime).

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Look at the next number that is not yet crossed out (3) which will be the 2nd prime, and repeat the process for that number.

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Look at the next number that is not yet crossed out (5) which will be the 3rd prime, and repeat the process for that number.
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Repeat until you find the k^th prime.

The algorithm in code

The first 25 primes are: 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, and 97.

Let us see how we would find them using the sieve of eratosthenes:

First, choose a SIZE greater than or equal to the largest prime you want. Choosing a SIZE too big means your algorithm will run slower. Choosing a SIZE too small means you won't find the n^th prime. This means you will need to know approximately how big the n^th prime is to make an array large enough. This is a problem to think about later however, let us use 100 for this example.
Now we can treat each index of the array as the value we are testing, and store a value to indicate prime or not.
Mark all of the numbers up to that SIZE. (you could do counting numbers instead if you like)
Initialize the array so all values are prime, then mark 0 and 1 as composite.
Find the next index that is prime, Now flag all indices that are the multiples of that as not prime.
Continue the process until you get a stopping condition. Either you got to the end of the array, or you found the n^th prime.

Lab04 Requirements

Clone the repo: https://classroom.github.com/a/SBdHcF-w

You will need to worry about sieve.c and sieve.h.

Critical: You will put a main in some other c file, not in sieve.c . I will test your sieve.c with my own tester.

Suggestion: have your sieve() function call a different function with all the work in it. This makes it easy to update to whichever version you made that works fastests.

In sieve.c have a function int sieve(int n); that returns the nth prime.

sieve(1) returns 2
sieve(2) returns 3
sieve(25) returns 97
etc.

Optimizations:

Please make sure you make new versions as you try to add new features. Always keep your fastest working version around.

Note: When crossing out the multiples of eac prime, you MUST increment by that primes in the loop, not by 1. e.g. increment by 5 to cross out multiples of 5. This is not an optimization.

Here is a possible progression of optimizations.

No Optimizations, just malloc/calloc the array and get large primes. (Just make it work)
Estimate the size of your array using math (see prime approximation below)
Use char array instead of int
Only cross out multiples of primes IF the prime is less than the sqrt of the SIZE. This makes the entire process stop when you reach a high enough prime.
Try to cut down on your extra variables and extra function calls
Store only odd values in your array, this cuts down array traversal and memory allocation quite a bit. e.g. index 0 is 1, index 1 is 3, index 2 is 5 etc.
Try using bitwise operations and store bits (exploration of this topic is optional)

Note: Everyone should be able to do all of the non-bitwise optimizations!

Prime approximations

The prime number is approximately n * ln(n) by the prime number theorem. However this is not always >= the n'th prime. You will need to add a coefficient or offset.

Versions

You will need to make multiple versions of this method!

Make sieveV1(n) and comment what optimizations you added.

Make sieve(n) call your fastest working sieveVx() command.

e.g.

int sieve(int n){
   return sieveV5(n);
}

//basic version
int sieveV1(int n){
  //code
}

//stopped early when checking primes
int sieveV2(int n){
  //code
}
...

Testing

You need to test for both speed and correctness. You only have to account for sieve(n) where 1 <= n <= 2,000,000

Here is a quick test script to see how fast your version is:

#include <stdio.h>
#include "sieve.h"
int main(){
  //This quickly checks a variety of sizes and values
  //Do not try to exploit the fact that these function calls are in the same main. I run multiple tests.
  int input[] = {1, 2, 3, 4,  5,  6,  7,  8,  9, 10, 25,
      100, 1000, 2350, 2945, 9822,  10000,  50000, 73234,  100000,
      500000, 1000003,1510041,1932457, 2000000};
  int ans[] =   {2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 97,
      541, 7919, 20897, 26849, 102563, 104729, 611953, 926623,1299709,
      7368787, 15485927, 24049567, 31285333 ,32452843};

  for(int i = 0; i < sizeof(input)/sizeof(int); i++){
      int guess = sieve(input[i]);
      //printf("Test sieve(%7d)\t%s, your guess : %8d \n",input[i],(guess==ans[i])?"PASS":"FAIL",guess);
      if(guess!=ans[i]){
          //exit at first failed case.
          printf("ERROR: sieve(%7d) your guess : %8d \n", input[i], guess);
          return 1;
      }

  }
  printf("Pass all cases.");
  return 0;
}

Test results: (on same computer)

8.9 sec : Unoptimized (array always 32452844)

2.5 sec : Tight array bounds.

1.6 sec : Tight array bounds /w char array.

0.6 sec : Tight array bounds /w char array only storing odd values in the array.

[back to top of page]

2024-10-09

2024-10-09T12:00:00+00:00

2024-10-09

Strings in C

In c, strings are character arrays.

There is nothing special about the way character arrays work. Because strings are so useful, there are a few features of C that make working with them simpler.

By convention the last entry in a string character array is the NULL character (either 0 (the number), or '\0' (the character). This is not something that is guaranteed, if you want to create a string, you will need to make sure that there is a terminating NULL, if not, a number of string related functions will not work.

When you use `""` to make a string literal :

A character array large enough to store the string, including a terminating NULL, is created in memory.
The characters of the string are stored in that array, and a terminating NULL is added.
String literals are immutable
If a string literal is exactly repeated in code, a new character array is not created, instead, the orginal array is used. This means all references to the same immutable string literal refer to the same piece of memory.

Declaring Strings

There are 4 ways to declare strings in C (in eaxch example, the numbers and strings used are randomly chosen, none have special meaning in C).

char s[256];
- Declares a mutable array of 256 bytes.
- No speciic characters are saved to memory.
- No guarantee of a NULL character at any position.
char s[256] = "Imagine"
- Creates the immutable string literal "Imagine" .
- Declares a mutable array of 256 bytes.
- Copies the string "Imagine" , including a terminating NULL, into the first 8 bytes of the array s .
char s[] = "Tuesday";
- Creates the immutable string literal "Tuesday"
- Creates an 8 bytes array, large enough for "Tuesday" and a terminating NULL for the variable s
- Copies the string "Tuesday" , including a terminating NULL, into the array s
char *s = "Mankind";
- Creates the immutable string literal "Mankind"
- s becomes a pointer to that immutable string.
- It is important to note that in the last example, an array is not created. In that case s is just a pointer to the memory location with the immutable string lives. If you want a mutable string, you cannot declare it this way.

Examples:

#include 
int main() {
  char s[16];
  char s2[16] = "Imagine";
  char s3[] = "Fish";
  char *s4 = "Imagine";
  char *s5 = "Imagine";

  //printf("%s\n",s);//DO NOT PRINT UNINITIALIZED MEMORY!
  printf("%s\n",s2);
  printf("%s\n",s3);
  printf("%s\n",s4);
  return 0;
}

Working With String Variables

Everything we've covered about pointers and arrays still holds true, string variables are pointers, either array pointers or normal pointers.

It is important to keep track of variables vs. values .

char s[10] = "Trogdor!!";
- In this example, s is an array variable that points to the 10 byte array allocation. s .
- s is immutable, it cannot point to any other memory location. e.g. s = "is a dragon"; is an error.
- The values in the array s points to are not immutable. You can change the value of the string at any point. s[0] = 'M'; is perfectly good.
char *s = "No sports references";
- Here, s is a pointer.
- As a pointer, you can change the value s points to. s = "The Best"; is valid.
- Since s points to an immutable string literal, you cannot change the value of the string. s[0] = 'N'; is an error .

String library

You can find built in functions for handling strings in string.h

You can use man 3 string to get a summary of methods, or use a

[back to top of page]

2024-10-10

2024-10-10T12:00:00+00:00

2024-10-10

Lab05

Deadline October 16th 8am

You will have 3 lab days + homework time for this lab.

First, acquaint yourself with the basic string functions: strlen, strcpy, strcat, strncat and strncpy. You can look at the man pages for each if you are unclear as to their uses. I encourage you to play around with each so you are comfortable with how to use them.

Then write your own versions of the following:

strlen: int mystrlen( char *s )
strcpy/strncpy: char * mystrcpy( char *dest, char *source ) and char * mystrncpy( char *dest, char *source, int n)
strcat/strncat: char * mystrcat( char *dest, char *source ) and char * mystrncat( char *dest, char *source, int n) ]
strcmp: int mystrcmp( char *s1, char *s2 ). Your version does not need to return the same exact value as the system version, as long as it returns -, + or 0 when it should.
strchr: char * mystrchr( char *s, char c )

Required use the function headers provided above, they may differ from the standard headers.

You must test every function you write, you will not get full credit for an untested functions.

Your program should consist of:

the main.c file that I have prepared for you, available in the github classroom lab05 repo
a mystring.c mystring.h pair of files for your string functions.
a makefile with a compile target as the default, and a run target.

When you test your functions, run them alongside the regular ones so you can see if they behave the same way or not. Remember you should call them my+originalName.

You must look at documentation to determine the behavior of the functions, in particular the n vs non-n versions!

Use this repository for the work:

https://classroom.github.com/a/MMknZczF

When you are done, you should look at the following two functions:

1. strstr : returns a pointer to the first index of a specified substring in another string.

char * mystrstr( char *s1, char * s2 )

2. mysplit

char * mysplit(char *s, char d) This is not quite a built in string function, but similar to strsep. This function should go through s, replacing one instance of c with a NULL byte. It returns a pointer to the byte after the null

Why is this useful? How can it be used?

Attempt to write these functions.

Here's an example of what your code might look like when run:

start strings:
s1: []
s2: [hello]
s3: [goodbye]

Testing strlen(s1):
[standard]:     0
[mine]:         0

Testing strlen(s2):
[standard]:     5
[mine]:         5

Testing strcpy(s1, s2):
[standard]:     [hello]
[mine]:         [hello]

Testing strncpy(s1, s3, 3):
[standard]:     [goo______________________________________________]
[mine]:         [goo______________________________________________]

Testing strcat(s1, s3):
[standard]:     [hellogoodbye]
[mine]:         [hellogoodbye]

Testing strncat(s1, s2, 3):
[standard]:     [hellohel]
[mine]:         [hellohel]

Testing strchr(s1, 'l'):
[standard]:     [0x7ffff3e10112]
[mine]:         [0x7ffff3e10112]

Testing strchr(s1, 0):
[standard]:     [0x7ffff3e10115]
[mine]:         [0x7ffff3e10115]

Testing strchr(s1, 'z'):
[standard]:     [(nil)]
[mine]:         [(nil)]

Testing strcmp
Comparting ab to abc:
        [standard]:     [-1]
        [mine]:         [-99]
Comparting abc to ab:
        [standard]:     [1]
        [mine]:         [99]
Comparting abc to abc:
        [standard]:     [0]
        [mine]:         [0]

[back to top of page]

2024-10-15

2024-10-15T12:00:00+00:00

2024-10-15

Results of Sieve

Here are the top 3 student times relative to mine:

Name	Time	% of K
09.Chen.Matthew	1.73	.81
10,Yu.Kellen	1.9	.89
K	2.12	1
09.Liang.Edmund	2.5	1.17
10.Chen.Alan	3.09	1.45
09.Goihman.Benjamin	3.125	1.47
09.Hasan.Tanzeem	3.189	1.50

Compared to 2023's top 3 students:

Name	Time	% of K
K	0.185	1
10.rahman.naowal/	0.209	1.13
08.li.andrew/	0.332	1.79
09.winer.charlie/	0.333	1.8

Compared to the 2022's top 4 students:

Name	Time	% of K
3.dewan.zawad	0.532	0.89
K	0.599	1
hughes.vernon	0.686	1.15
eisenberg.lee	0.701	1.17
paltrowitz.jacob	0.706	1.18

C can do some funky things:

If your strcpy looks like this:

char* mystrcpy(char* d, char* s){
  char* orig = dest;
  while(*src){
    *dest=*src;
    dest++;
    src++;
  }
  *dest=*src;
  return orig;
}

Consider instead how this would work:

char* mystrcpy(char* d, char* s){
  char* orig = d;
  while(*d++ = *s++);
  return orig;
}

Or the ghastly:

char* mystrcpy(char* d, char* s){
	return (*d=*s) ? mystrcpy(d+1,s+1) - 1 : d;
}

struct

A struct is a custom data type that is a collection of values.

Declaring

The following line creates a variable, s, who's type is an anonymous struct:

struct { int a; char x; } s;

struct { int a; char x; } is the full type of s, it is syntactically identical to int or float

Utilization

We use the . operator to access a value inside a struct

s.a = 10;

s.x = '@';

Here is an example of creating and using a struct:

 int main() {
  struct {int a; char x;} s0;

  s0.a = 51;
  s0.x = '%';

  printf("s0: %d\t%c\n", s0.a, s0.x);

  return 0;
}

Creating prototypes

It is preferable to prototype your structs, which will make it easier to create and work with multiple variables of the same struct type.

struct foo { int a; char x; };
Note that since we are not creating a variable, there is no name between the } and the ; at the end.

After creating a prototye for a struct, you can declare new variables of that type like so:

struct foo s;
You still must include the word struct.

It is typically better practice to prototype structs outside of any particular function.

Struct prototypes are most commonly found in .h files.

Example

Here is an example of creating and using a struct with a prototype:

struct foo {int a; char x;};
int main() {
   struct foo s0;
   struct foo s1;

   s0.a = 51;
   s0.x = '%';

   s1 = s0;
   printf("s0: %d\t%c\n", s0.a, s0.x);
   printf("s1: %d\t%c\n", s1.a, s1.x);

   return 0;
}

Pointers and Structs

You can make pointers to structs like pointers to primitaves.

struct foo *p = &s;

DANGER!

One very important note, . takes precedence over *.

This means that *p.x is the same as *(p.x) which is almost certainly NOT what you want. (This will look for x inside p and de-reference that result).

To access a value in a struct via a pointer you need to do: (*p).x, that is, de-reference first, then get x.

Shortcut:

In C, p->x is syntactic shorthand for (*p).x

[back to top of page]

2024-10-16

2024-10-16T12:00:00+00:00

2024-10-16

Struct lab

This is a small part of the lab that doesn't need that much time. (It is intended to be rather basic, if you have trouble let me know.)

Due: Tomorrow 10/17 In class! (Just this much, you will expand on it in class this is just the basics.

lab06

GitHub Submission Link: https://classroom.github.com/a/nlVP82jI

Create a makefile with compile and run targets.
Create a mystruct.c and mystruct.h file for your struct methods, and a separate main.c file for your main. (This should be default style behavior for all of your projects).
Create a struct movie that has 3 data members, int, float, and a string. (Use a char[] NOT char* for this string.)
Let us put in Strings that are the names of movies. The int can be the year released, double can be the average rating of that movie out of 10.
Write a function that print(struct movie m) that prints your struct in a single line.
Write a function that createmovie(...) creates a new struct in heap memory and returns its address. It should have paramaters appropriate to your struct.
Write your main function that:
- Create an array of 4 pointers to your struct.
- Populate it with randomized data using your creating method.
- Print your data in a reasonable way (one element per line)

hint: You may need to make an array of strings so you can choose one randomly see note below.

char arr[3][10] = {"abc", "abcd", "abcde"};
printf("String array Elements are:\n");
for (int i = 0; i < 3; i++)
{
  printf("%s\n", arr[i]);
}

[back to top of page]

2024-10-18

2024-10-18T12:00:00+00:00

Lab06-structs is really linked-lists

Due: Monday 10/21 8:00 am

New Things to add to the lab

Write a simple linked list library + program.

Your labs should always be compileable via compile and run targets of your makefile
Rename your myStruct.c/.h from the previous instructions to node.c/.h
Change your struct name to struct node
Add a new element to the struct: A pointer to a struct of the same type (the next node).
Write several list functions outlined below. They should be in a list.c/.h library. You must have a separate c file used for your main function and testing.
Your main should print a series of tests for these various functions. This should be enough to convince yourself (and me) that the functions work correctly*. I would advise you to test removing first, last and middle elements, multiple times in case one of these cases breaks future functionality.
*Correctly does not include: warnings or segementation faults.

Here's an example of how a struct could have a pointer to the same type inside of it:

struct user_node {
  int id;
  char name[200];
  struct user_node *next;
  };

Teacher Testing

I will create a driver main program, that includes your list.h and node.h then uses the functions. If this does not work, you will not recieve credit.

Create the following functions:

void print_list(struct node * list)
- Should take a pointer to a node struct and print out all of the data in the list, one element per newline. Prefix each line with the index of the element e.g.
```
[0]Star Wars: Episode IV - A New Hope (1977) 8.6
[1]Alien (1979) 8.5
[2]The Matrix (1999) 8.7
[3]Inception (2010) 8.8
[4]Interstellar (2014) 8.7
```
- You should call your print() function from your node class. Modify it a little so that the formatting is the same as the example: "The Matrix (1999) 8.7" or "TITLE (YEAR) SCORE" with spaces between them.
struct node * insert_front(struct node * list, char* title, int year, double rating)
- Should take a pointer to the existing list and create a new node and put it at the beginning of the list.
- The second argument will be an char* that contains a string that should be copied into your node.
- The numerical parameters should also be copied into your node.
- Returns a pointer to the beginning of the list.
struct node * free_list(struct node * list)
- Should take a pointer to a list as a parameter and then go through the entire list freeing each node and return a pointer to the beginning of the list (which should be NULL by then).
struct node * remove_node_by_index(struct node * list, int index)
- Remove the node in the index position, from the list pointed to by front. Assume this list starts at index zero.
- A node that is removed must be free'd
- If index is negative or index >= the length of the list, nothing is changed. (This includes when the list is empty)
- Returns a pointer to the beginning of the list.

Sample Usage

struct node * movies = NULL;
movies = insert_front(movies, "Pulp Fiction", 1994, 9.0);
movies = insert_front(movies, "Battlefield Earth", 2000, 2.5);
print_list(movies);
movies = free_list(movies);

You should recognize that functions that return a pointer to the beginning of the list should be assigned to the list, like insert, or even free!

[back to top of page]

2024-10-21

2024-10-21T12:00:00+00:00

2024-10-21

File functions

open - <fcntl.h>

Add a file to the file table and returns its file descriptor.
This will make the file accessible within a program via the returned file descriptor.
If open fails, -1 is returned, extra error information can be found in errno.
- errno is an int variable that can be found in <errno.h>
- Use strerror (in string.h) on errno to return a string description of the error
open( path, flags, mode )
- mode
  - Only used when creating a file. Set the new file's permissions using a 3 digit octal #
- flags
  - Determine what you plan to do with the file, use the following constants and combine with |:
  - O_RDONLY
  - O_WRONLY
  - O_RDWR
  - O_APPEND
  - O_TRUNC
  - O_CREAT
  - O_EXCL: when combined with O_CREAT, will return an error if the file exists
- examples:
  - open(foo.txt, O_RDONLY, 0)
  - open(goo.txt, O_WRONLY | O_APPEND | O_CREAT, 0644)

read - <unistd.h>

Read data from a file
read( fd, buff, n )
- Read n bytes from fd's file into buff
- Returns the number of bytes actually read. Returns -1 and sets errno if unsuccessful.
- buff must be a memory address (pointer or array), but can be to any type of data.

write - <unistd.h>

Write data to a file
write( fd, buff, n )
- Write n bytes to the fd's file from buff
- Returns the number of bytes actually written. Returns -1 and sets errno if unsuccessful.
- buff must be a memory address (pointer or array), but can be to any type of data.
- lseek - <unistd.h>
  - Set the current position in an open file
  - lseek( file_descriptor, offset, whence )
    - offset
      - Number of bytes to move the position by, Can be negative.
    - whence
      - Where to measure the offset from
      - SEEK_SET: offset is evaluated from the beginning of the file
      - SEEK_CUR: offset is relative to the current position in the file
      - SEEK_END: offset is evaluated from the end of the file
  - Returns the number of bytes the current position is from the beginning of the file, or -1 (errno)

stat - <sys/stat.h>

Get information about a file (metadata)
stat( path, stat_buffer )
- stat_buffer
  - Must be a pointer to a struct stat
  - All the file information gets put into the stat buffer.
  - Some of the fields in struct stat:
    - st_size: file size in bytes
    - st_uid, st_gid: user id, group id
    - st_mode: file permissions
    - st_atime, st_mtime: last access, last modification
      - These are time_t variables. We can use functions in time.h to make sense of them
      - ctime( time )
        
        Returns the time as a string
        
        time is type time_t *

opendir - <dirent.h>

Open a directory file
This will not change the current working directory (cwd), it only allows your program to read the contents of the directory file
opendir( path )
- Returns a pointer to a directory stream (DIR *)
closedir - <dirent.h>
- Closes the directory stream and frees the pointer associated with it.
- closedir( dir_stream )
readdir - <dirent.h>
- readdir( dir_stream )
- Returns a pointer to the next entry in a directory stream, or NULL if all entries have already been returned.
struct dirent - <sys/types.h>
- Directory struct that contains the information stored in a directory file. Some of the data members
- d_name: Name of a file
- d_type: File type as an int
- Use man readdir to list the different components.
- Example usage:
```
DIR * d;
d = opendir( "somedir" );
struct dirent *entry;
entry = readdir( d );
closedir(d);
```
rewinddir - <dirent.h>
- rewinddir(d)
  - d must be a DIR * returned from opendir
  - Resets the directory stream of d to the beginning.

[back to top of page]

2024-10-22

2024-10-22T12:00:00+00:00

2024-10-22

Demo Code To Be Discussed:

#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
#include <string.h>
#include <sys/stat.h>
#include <errno.h>
#include <time.h>
#include <sys/types.h>
#include <dirent.h>
int err(){
    printf("errno %d\n",errno);
    printf("%s\n",strerror(errno));
    exit(1);
}

#define BUFFER_SIZE 128

int main(){
  int r_file,w_file;
  char buff[BUFFER_SIZE+1];
  buff[BUFFER_SIZE]=0; // for printf purposes.

  /////////////////////////////////////
  //Fail to open:
  //Either don't have the file, or grant no read access
  r_file = open("./cantouchthis", O_RDONLY , 0);
  printf("open(./canttouchthis) : %s\n",strerror(errno));

  ////////////////////////////////////
  //Open for read
  r_file = open("./text.book", O_RDONLY , 0);
  if(r_file == -1) err();
  printf("r_file: %u\n",r_file);

  ////////////////////////////////////
  //Open for write
  w_file = open("./copy",
      O_WRONLY | O_APPEND | O_CREAT, 0611);
  if(w_file==-1)err();
  printf("w_file: %u\n",w_file);


  ////////////////////////////////////////
  //READ and WRITE
  int bytes;
  //putting the assignment in the while boolean
  //eliminates the need to check for 0 bytes read
  while(bytes = read(r_file, buff, BUFFER_SIZE)){
    if(bytes == -1)err();//all non 0 are true

    printf("Bytes read: %u",bytes);
    printf("'''%s'''\n",buff);

    //write the bytes
    int wbytes = write(w_file , buff, bytes);
    if(wbytes == -1)err();;


  }


  /////////////////////////////
  //WRITE NUMBERS!
  int x = 97;
  //write the 4 bytes of the int, 97,0,0, and 0.
  write(w_file,&x,4);

  x= 65+25;//Z
  //Write the 1st byte of x (least significant)
  write(w_file,&x,1);

  double y = 1234.5678;
  //Write a double... this will end poorly.
  write(w_file,&y,8);//write a double

  ///////////////////////////////////////
  //STAT!
  struct stat * stat_buffer;
  struct stat s_buff;//not used in the example

  //malloc sizeof the struct, not the pointer!
  stat_buffer = malloc(sizeof(struct stat)*1);

  stat( "./copy", stat_buffer);//or use &s_buff
  printf("size: %ld \n", stat_buffer->st_size);
  printf("mode : %o \n", stat_buffer->st_mode);

  time_t t = stat_buffer->st_mtime;
  printf("%s\n",ctime(&t));
  free(stat_buffer);


  //////////////////////////////////////
  //DIRENT
  DIR * d;
  char* PATH = ".";
  d = opendir( PATH );
  struct dirent *entry;

  //Can use as a while boolean
  //since null is the end of the dirent list.
  while(entry = readdir( d )){
    printf("filename: %s\n",entry->d_name);
    printf("--type: %d\n",entry->d_type);
    //You can use the PATH + filename to get stat data!
  }
  closedir(d);
}

[back to top of page]

2024-10-23

2024-10-23T12:00:00+00:00

Pair Project:

Choose a partner for the first project (music playlist manager using linked lists)

We will start the project on Monday.

You should check if both you and your partner know how to handle:

Branching.
Merging.
Merge conflicts.

/dev/random

Your computer measures temperatures and voltages and generates an 'entropy pool' to create truly random numbers.

You can read bytes from /dev/random to return random bytes from the 'entropy pool'.

When the entropy pool is empty (you read too many values), reads from /dev/random will block until additional environmental noise is gathered.

/dev/random is be suitable for uses that need very high quality randomness such as one-time pad or key generation.

/dev/urandom

When you do NOT need totally cryptographically secure values, or you cannot have your program block we read from /dev/urandom

A read from the /dev/urandom device will not block waiting for more entropy. As a result, if there is not sufficient entropy in the entropy pool, the returned values a

Lab07-Files

dev[ise a]/random plan

Check the File Functions section of the notes page for reference.

Repo Link: https://classroom.github.com/a/aQupXEVS

Read or write

Your goal for this lab is to create a program that can either write to or read from a file. You can do this by using command line arguments or making two separate main programs. You would need to look up command line parsing in c, so making 2 main programs is acceptable.

Your makefile will have 2 run targets. If you have 2 main files, your compile target can compile both at the same time.

Requirements

A function that generates and returns a random int by reading in the correct number of bytes from /dev/random . Please note that this is a file for purposes of reading. This should be in a separate c/h library!
A main function (or two) that can either WRITE values to a file, or READ values from a file.

make write
1. Declare two int variables and an int array of size 8
2. Populate the two variables and an array with random numbers generated by your random function.
  
  If your program seems to stall at this step, it's possible that your computer doesn't have enough entropy and is waiting, you can read from /dev/urandom instead.
3. Prints out the contents of the two variables and array.
  - Write the two variables to the file.
  - After testing the two variables, also write the array to a file called arraydata.dat. Just use a single write command (DO NOT use a loop).
make read
1. Declare two int variables and an int array of size 8
2. Reads the data from the file into your program.
  - Read the two random values from the file into two new variables.
  - Read the whole array from the file into a different array using a single read command. (DO NOT use a loop!)
3. Prints the contents of the two variables/array.
Urgent: This program should work if there isn't already a arraydata.dat, so it should be able to create the file when it doesn't exist. I will be rm -f arraydata.dat in my grading script, so I know your code works when the file isn't already created.

Remember to use good practices, like checking return values for errors

Also remember to include a makefile that includes separate compile + read + write targets

Sample output (yours will vary)

make write
    Generating random numbers:
    x : 12875823
    y : 40258233
    ary[0]: 198116400
    ary[1]: 259268753
    ary[2]: 283364368
    ary[3]: 327370590
    ary[4]: 354867368
    ary[5]: 387688898
    ary[6]: 261733183
    ary[7]: 172096907
  Writing numbers to file...

make read
  Reading numbers from file...
    x : 12875823
    y : 40258233
    ary[0]: 198116400
    ary[1]: 259268753
    ary[2]: 283364368
    ary[3]: 327370590
    ary[4]: 354867368
    ary[5]: 387688898
    ary[6]: 261733183
    ary[7]: 172096907

[back to top of page]

2024-10-28

2024-10-28T12:00:00+00:00

2024-10-28

MyTunez™

Logistics

Partner

You may work with a partner on this assignment. They must be in the same class period as you.

Working in a group requires the use of branching, see the tutorial here: Git Branching Tutorial

Repo

The github classrooms link works differently with groups. The first partner in a group should create a new team when accepting the assignment, then the 2nd person should join that team.

Only join a team if the other person agreed, otherwise your teacher must manually change the teams.

DO NOT JOIN UNTIL CLASS DISCUSSION assignment repo: https://classroom.github.com/a/osoZpCnK

Team name should be PERIOD-LastnameFirstInitial-LastName2FirstInitial2

E.g. this group of 3 students in period 5: 5-WrightG-WongL-WongJ

You can put a creative group name in your readme, e.g. Two Wongs Don't Make A Wright

Music Library Overview

Write a program in C that implements a music library organizer.

This will consist of 2 data structures:

A linked list of songs in node.c/h
- Your nodes/links should be able to hold a song's title and artist. You should use structures like the following:
```
              struct song_node{
  char artist[100];
  char title[100];
  struct song_node *next;
};
```
The actual library in library.c/h. This should be an array of 27 linked lists (one for each letter from 'a' to 'z', and another for symbols).
- Each slot will contain a linked list of all the artists that have names that start with the corresponding letter.
- When you add a song, it should go on to a linked list at the appropriate array slot, and then determine the correct position alphabetically, first by artist, then by song.

Linked List Part:

You should start by making your linked lists work with the following functionality:

insert song at the front (this is just for testing)
insert song in alphabetical order (this is the primary way to insert for later)
- Alphabetical is by Artist then by Song, so all ACDC songs come before all of the Black Sabbath songs
- You must make a int compare(struct song_node * a, struct song_node * b) functions to easily compare song nodes effectively. These comparisons should be case INSENSITIVE, so: A &lt Abba &lt ACDC &lt Aerosmith. Abba and abba would be considered equal. You can make a lowercase version of the strings before you compare, or convert the numbers as you compare them. (anything from 97(a) to 122(z) inclusive is counted as 65(A) to 90(Z). )
void print_song_list(struct song_node *) print the entire list, see format in examples.
find and return a pointer to a node based on artist and song name
find and return a pointer to the first song of an artist based on artist name
Return a pointer to random element in the list.
remove a single specified node from the list
- specified by both artist and song name.
free the entire list, anything that was malloc'ed must be freed

Library Part

DO NOT WORK ON THIS PART UNTIL YOUR LINKED LIST IS COMPLETE

Then create your array of linked lists (suggested type: struct song_node ** for the full program to have the following functions:

struct song_node ** init(); initialize a new library: Allocate enough memory for 27 linked lists, make sure each entry is an empty list.
void add(struct song_node ** library, char* artist, char* title) Add song alphabetically to the correct slot in the library.
struct song_node * search_song(struct song_node ** library, char* artist, char* title ) Search for a song given song title and artist (return a pointer, or NULL if it isn't found).
struct song_node * search_artist(struct song_node ** library, char* artist ) Search for the first song that is by the provided artist (return a pointer, or NULL if it isn't found).
print_letter(struct song_node ** library, char letter) Print out all the entries under a certain letter. The letter var shoudl use capital 'A' through'Z', or '0' for non-letter artists.
print_artist(struct song_node ** library, char* artist) Print out all the songs of a the provided artist, use the same format as a print_song_list. This may print nothing if no matching artist is found.
print_library(struct song_node ** library) Print out the entire library , see format in examples.
void shuffle (struct song_node ** library, int n) Shuffle - prints out a series of n randomly chosen songs. (can have duplicates) Please make sure the distribution of random is equal per song. If you have 100 ABBA songs, and 2 Beoncé songs, the ABBA songs should print 50x more frequently than Beoncé.
int delete_song(struct song_node ** library, char* artist, char* title ) Delete a song from the library, make sure the memory is free'd.
- specified by both artist and song name.
- Return 0 if the song was deleted, 1 if the song was not found.
void reset(struct song_node ** library) Clear out all the linked lists in the library. This must free() all memory malloc'ed. (Originally it had an int return, that was an error.)

Putting it all together

Files you should have:

A .c file that contains the main function you use to test all your work. You must test all the functions in your main function in order to receive full credit!!!
Two sets of .h/.c files. There are 2 separate structures being worked on, the linked list (node.c/h) and the library (library.c/h).
A makefile with a (default) compile target that generates a single executable program, as well as a run target.

Example testing

LINKED LIST TESTS
====================================

Testing print_list:
	[ {ac/dc, thunderstruck} | {pearl jam, alive} | {pearl jam, even flow} | {pearl jam, yellow ledbetter} | {pink floyd, time} | {radiohead, paranoid android} | {radiohead, street spirit (fade out)} | ]
====================================

Testing print_node:
	{ac/dc, thunderstruck}
====================================

Testing find_node:
looking for [pearl jam: even flow]
	node found! {pearl jam, even flow}
looking for [pearl jam: daughter]
	node not found
====================================

Testing find_artist:
looking for [pink floyd]
	artist found! [ {pink floyd, time} | {radiohead, paranoid android} | {radiohead, street spirit (fade out)} | ]
looking for [pearl jam]
	artist found! [ {pearl jam, alive} | {pearl jam, even flow} | {pearl jam, yellow ledbetter} | {pink floyd, time} | {radiohead, paranoid android} | {radiohead, street spirit (fade out)} | ]
looking for [presidents of the united states of america]
	artist not found
====================================

Testing songcmp (helper function):
Comparing [pearl jam: even flow] to [pearl jam: even flow]
	0
Comparing [pearl jam: even flow] to [pearl jam: alive]
	4
Comparing [pearl jam: alive] to [pearl jam: even flow]
	-4
Comparing [pearl jam: even flow] to [pink floyd: time]
	-4
====================================

Testing random:
{pink floyd, time}
{radiohead, paranoid android}
{radiohead, street spirit (fade out)}
{pearl jam, alive}
====================================

Testing remove:
Removing [ac/dc: thunderstruck]
	[ {pearl jam, alive} | {pearl jam, even flow} | {pearl jam, yellow ledbetter} | {pink floyd, time} | {radiohead, paranoid android} | {radiohead, street spirit (fade out)} ]
Removing [radiohead: street spirit (fade out)]
	[ {pearl jam, alive} | {pearl jam, even flow} | {pearl jam, yellow ledbetter} | {pink floyd, time} | {radiohead, paranoid android} ]
Removing [pearl jam: yellow ledbetter]
	[ {pearl jam, alive} | {pearl jam, even flow} | {pink floyd, time} | {radiohead, paranoid android} ]
Removing [pearl jam: yellow ledbetter]
	{pearl jam: yellow ledbetter} not found
[ {pearl jam, alive} | {pearl jam, even flow} | {pink floyd, time} | {radiohead, paranoid android} ]
====================================

Testing free_list
	freeing_node: {pearl jam, alive}
	freeing_node: {pearl jam, even flow}
	freeing_node: {pink floyd, time}
	freeing_node: {radiohead, paranoid android}
list after free_list:
[ ]
====================================

MUSIC LIBRARY TESTS
====================================

Testing print_letter
[ ]
====================================

Testing print_library
====================================

Testing print_letter
[ {pearl jam, alive} | {pearl jam, even flow} | {pearl jam, yellow ledbetter} | {pink floyd, time} | {presidents of the united states of america, peaches} ]
====================================

Testing print_library
a: [ {ac/dc, thunderstruck} ]
p: [ {pearl jam, alive} | {pearl jam, even flow} | {pearl jam, yellow ledbetter} | {pink floyd, time} | {presidents of the united states of america, peaches} ]
r: [ {radiohead, paranoid android} | {radiohead, street spirit (fade out)} ]
====================================

Testing find:
looking for [pearl jam: alive]
	song found! {pearl jam, alive}
looking for [pearl jam: time]
	song not found
====================================

Testing find artist:
looking for [pearl jam]
	artist found! [ {pearl jam, alive} | {pearl jam, even flow} | {pearl jam, yellow ledbetter} | {pink floyd, time} | {presidents of the united states of america, peaches} ]
looking for [pink floyd]
	artist found! [ {pink floyd, time} | {presidents of the united states of america, peaches} ]
looking for [bob dylan]
	artist not found
====================================

Testing remove_song
removing: [pearl jam: alive]
a: [ {ac/dc, thunderstruck} ]
p: [ {pearl jam, even flow} | {pearl jam, yellow ledbetter} | {pink floyd, time} | {presidents of the united states of america, peaches} ]
r: [ {radiohead, paranoid android} | {radiohead, street spirit (fade out)} | ]

removing: [pearl jam: yellow ledbetter]
a: [ {ac/dc, thunderstruck} ]
p: [ {pearl jam, even flow} | {pink floyd, time} | {presidents of the united states of america, peaches} ]
r: [ {radiohead, paranoid android} | {radiohead, street spirit (fade out)} ]
====================================

Testing clear_library:
	freeing_node: {ac/dc, thunderstruck}
	freeing_node: {pearl jam, even flow}
	freeing_node: {pink floyd, time}
	freeing_node: {presidents of the united states of america, peaches}
	freeing_node: {radiohead, paranoid android}
	freeing_node: {radiohead, street spirit (fade out)}

Library after clear:
====================================

Adding songs to empty library
a: [ {ac/dc, thunderstruck} ]
p: [ {pearl jam, alive} | {pearl jam, even flow} | {pearl jam, yellow ledbetter} | {pink floyd, time} ]
====================================

Testing print_artist:
Printing [pearl jam]
pearl jam: [ {pearl jam, alive} | {pearl jam, even flow} | {pearl jam, yellow ledbetter} ]

Printing [ac/dc]
ac/dc: [ {ac/dc, thunderstruck} ]
====================================


Printing [radiohead]
radiohead:
====================================

Testing shuffle
{pink floyd, time}
{pearl jam, yellow ledbetter}
{pearl jam, alive}
====================================

[back to top of page]

2024-10-30

2024-10-30T12:00:00+00:00

2024-10-30

Pclassic

PClassic's 2024 Fall competition is on Nov. 23, 2024. You can familiarize yourselves with the competition questions of previous years here : https://www.pclassic.org/problems

If you are interested you should make a team and practice solving the problems. You shoudl also actually practice together on a single computer if you are serious.

Information Document

StuyCCC has compiled a document of relevant information, including important dates and how to sign up for the in-person trip:

Sign up form (also on the document above)
Don't forget to join our Discord server and Epsilon for further details:
https://discord.gg/Wd2FxeE
https://epsilon.stuysu.org/stuyccc

2024-11-04

2024-11-04T12:00:00+00:00

2024-11-04

Input

Command Line Arguments:

int main( int argc, char *argv[] )

Program name is considered the first command line argument
argc is the number of command line arguments
argv is the array of command line arguments as strings

stdin input

fgets is in &ltstdio.h>

fgets will read in data from a file stream (I/O stream) and store it in a string. This is how we can read from stdin.
fgets( char * buffer, int n, FILE * f );
- FILE * is the file stream type, more complex than a file descriptor, allows for buffered input.
- Reads at most n-1 characters from file stream f and stores it in s, appends NULL to the end.
- Stops at newline, end of file, or the byte limit.
- Returns a pointer to the buffer, or null if no bytes were read.
- stdin is a FILE * variable
Example: fgets(s, 100, stdin)

Other file streams

The fopen() function is a higher level wrapper for the system call open() that returns a file stream. We can open files using fopen() to enable using fgets() on regular files.

fopen() returns FILE * (a file stream) attached to the specified file/device. If the function fails, it returns a null pointer.
fopen(char *path, char *mode) path is the same as open, but mode is now a string, and it supports: "r", "w", "a", "w+" (read, write, append, write but the file must exist). More mode combinations can be found in the documentation. (see man 3 fopen)

Pulling data from strings

sscanf in &ltstdio.h>

Reads in data from a string using a format string to determine types.
sscanf( char *s, char * format, void * var0, void * var1, ... )
Copies the data into each variable.

examples:

char* s = "103 1.23 0.042311234";
int x; float f; double d;
sscanf(s, "%d %f %lf", &x, &f, &d);

char dtm[32];
strcpy( dtm, "Saturday March 25 1989" );
sscanf( dtm, "%s %s %d  %d", weekday, month, &day, &year );
printf("%s %d, %d = %s\n", month, day, year, weekday );

Use your skills!

Write a program to help determine if triangles are valid.

The program will either take user input or parses a file named 'input' wtih the following file format : any plain text file with 3 integers on each line, that contains a multiple of 3 lines inside of it (but not 0).

After compiling, when you run your program, there are 2 calculations to make.

Determine how many triangles are valid. We will assume that valid just means you can make a triangle with the given side lengths.

./a.out user will read 3 ints from stdin and print either valid or invalid.

./a.out horizontal will treat each line of input as a triangle, and print the total number of valid triangles in the set.

(OPTIONAL CHALLENGE)./a.out vertical will treat each set of 3 lines as 3 vertically aligned triangles, and print the total number of valid triangles in the set. (This is more challenging but only because it gets ugly dealing with the 9 values in a 3 line set.)

./a.out with no argumetns, or with invalid arguments will print directions.

Note that given:

330  143  338
769  547   83
930  625  317

horizontal means the 1st triangle is 330 143 338

CHALLENGE: vertical means the 1st triangle is 330 769 930

Here is a larger test input:

330  143  338
769  547   83
930  625  317
669  866  147
15  881  210
662   15   70
273  277  707
50  592  770
280  313  407
642  487  372
94  619  295
734  243  141
220  141  515
198  824  745
210  787  964
330  143  338
769  547   83
930  625  317
669  866  147
15  881  210
662   15   70
273  277  707
50  592  770
280  313  407
642  487  372
94  619  295
734  243  141
220  141  515
198  824  745
210  787  964

[back to top of page]

2024-11-05

2024-11-05T12:00:00+00:00

2024-11-05

Stuy Quantum

StuyQuantum is holding its very own Qiskit Fall Fest in partnership with IBM Quantum from Thursday, November 7th at 4:30 pm to Thursday, November 14th at 5:30 pm! Join us this Thursday for instructions and the kickoff event! If you can not make it, all instructions will be available on our website! It is not necessary to be in the club to participate, but definitely consider joining! Our website also has our lesson slides in case you want to review any topics. The old lesson slides are password protected, so email either Matthew Karp or Jay Chen for the password. Join our discord to discuss and ask questions about the event. No prior experience is necessary! Any submission will receive a certificate of completion if effort is shown!

stuyquantum.com/fallfest.html

Join our social media (Discord, Insta, Epsilon) found on DiscordLink for more updates!

Lab08 Directory info

Deadline is Tuesday 8am., You will be assigned a new lab on Tuesday!

Write a program that will gather various information about a directory.

This will use your args parsing , as well as your ability to use dirent and stat().

Functionality:

By default the target is the current directory ("./")

list all the files in the directory
specify which files are directories (if any)
show the total size of all the regular files the directory. Do NOT include the size of folders (they show as 4096bytes). You can use du -h -d 1 to see the total size of the current directory including all subdirectories.
- note that you do not need to recursively go through any subdirectories to find their size for this part (unless you want to, but that is not a simple task)
As always you should have compile and run targets in your makefile

Possible enhancements:

Recursively traverse sub directories and calculate the size of the files in those subdirectories. Display the current directory total, AND the total including all sub directories (do not traverse . or .. ) .
Print out the size in a more readable format (like using KB, MB, GB for -byte prefixes)
Make it look more like ls -l by including permissions

User Input:

After you complete the above portion your program should handle user input as follows:

If a command line argument is entered, use that as the directory to scan, otherwise use "./"
If the argument does not have a trailing / you may wish to add one.
if an invalid directory is entered or you do not have permission, print a reasonable error message. (hint: you must use errno)
In your makefile you can use $(ARGS) to represent your command line arguments. So:
If your recipe says: ./a.out $(ARGS)

Then you call make as follows: make run ARGS="PATH_GOES_HERE"

Make would then run: ./a.out PATH_GOES_HERE

Testing:

Make a few different directories with some files in them.

Your make run should work on the current directory, which should be the same as make run ARGS="./"

You should test many different cases for the path: make run ARGS="folder1/" , make run ARGS="../folder3/" , and make run ARGS="folder1/folder2/"

Repository

Create Your Lab08 repo

Example:

Below is sample output, yours may differ slightly, but as long as it presents the information requested above you will receive full credit.

Please list the directories first, alphabetically is not required.

$ make run
  Statistics for directory: .
  Total Diectory Size: 17766 Bytes
  Directories:
        .
        ..
        dir
        structRW
  Regular files:
        a.out
        error.log
        filelist
        files.c
        read.c
        ReadTest
        redirect.c
        redirect.c~
        redirect_full.c
        redirect_full.c~
        redirTest
        statfiles.c
        statfiles.c~
        write.c
        WriteTest

[back to top of page]

2024-11-12

2024-11-12T12:00:00+00:00

2024-11-12

strtok() is tokxic

The limitation of strtok(char *str, const char *delim) is that it can't work on multiple strings simultaneously as it maintains a static pointer. This is a hidden state, it is not a pure function.

You should never use strtok in a library because your library calls will break any existing strtok() usage.

strsep() to the rescue

strsep is in &LTstring.h>

Strsep will parse a string with a common delimiter : strsep( source, delimiters )

Locates the first occurrence of any of the specified delimiters in a string and replaces it with NULL

delimiters is a string, each character is interpreted as a distinct delimiter.

Returns the beginning of the original string, sets source to the string starting at 1 index past the location of the new NULL

Since source's value is changed, it must be a pointer to a string (char **).

example:

char line[100] = "woah-this-is-cool";
char *curr = line;
char * token;
token = strsep( &curr, "-" );

The code above:

replaces the '-' after woah with NULL
returns a pointer to the w in "woah"
sets curr to point to the t in "this-is-cool"

Lab09 Struct Files

Deadline: Monday 2024/11/18 8am

You should have a makefile with compile target that creates the correctly named program.

GH Classroom link: https://classroom.github.com/a/Bb1mCg4X

Requirement: Your make compile should output an executable named "structrw" for ease of scripting on my end. That is compile must use -o structrw

You will find a csv file of NYC census population values here: nyc_pop.csv Place this in the same directory as your code/makefile but don't add it to your repo.

For testing, here is another file: us_pop.csv

The header of the file is this: Year,Manhattan,Brooklyn nine-nine,Queens,The Bronx,Staten Island
An example line of data is this: 1790,33131,4549,6159,1781,3827
You should support comma separated headers of boroughs with spaces in them. (e.g.) Staten island does not have to be the last entry. There will always be 6 columns in my test cases.
You must use strsep() to parse the header. There are multiple ways to do this but I want you to learn strsep() .
There will always be 5 locations plus the year per line, so you can use sscanf to get the numbers.
Additionally you should probably terminate the header string by replacing the first '\n' (or '\r') with a 0. This will remove the newline from your strings, and handle windows and unix line endings.

Your goal: write a program that will perform the following actions based on a command line argument:

-read_csv
- Read the contents of the nyc_pop.csv file (this is text).
- Store each piece of data using the following struct (exactly) :
```
struct pop_entry {
  int year;
  int population;
  char boro[20];
};
```
- Create & write a new file nyc_pop.dat containing the population data using struct pop_entry . This file should contain byte data.
- If the file already exists, overwrite the existing one.
-read_data
- Read the contents of the data file into an array of struct pop_entry values. This array should be dynamically allocated based on the file size.
- Display the data in human-readable format, provide a number for each data entry when displaying.
-add_data
- Ask the user for new data to put into a struct pop_entry value.
- The data should be entered to stdin on one line in the format: Year population borough. (see note at end)
- Append the entry in the data file (not the csv).
-update_data
- Start by showing the dat file just like -read_data .
- Then prompt the user to enter the corresponding number of an entry to modify.
- Then prompt the user for new data.
- The data should be entered to stdin on one line in the format: Year population borough.(see note at end)
Update the entry in the data file (not the csv).

Sample output:

-read_csv

        $ ./structrw -read_csv
reading nyc_pop.csv
wrote 2760 bytes to nyc_pop.data

-read_data

This is only the fist 11 entries, there should be 115 in nyc_pop.csv at the start.

        $ ./structrw -read_data
0: year: 1790	boro: Manhattan	pop: 33131
1: year: 1790	boro: Brooklyn nine-nine	pop: 4549
2: year: 1790	boro: Queens	pop: 6159
3: year: 1790	boro: The Bronx	pop: 1781
4: year: 1790	boro: Staten Island	pop: 3827
5: year: 1800	boro: Manhattan	pop: 60515
6: year: 1800	boro: Brooklyn nine-nine	pop: 5740
7: year: 1800	boro: Queens	pop: 6642
8: year: 1800	boro: The Bronx	pop: 1755
9: year: 1800	boro: Staten Island	pop: 4563
10: year: 1810	boro: Manhattan	pop: 96373

-add_data

2020 Bronx 1390450 Was entered into stdin as a single input at the time of the program running.

        $ ./structrw -add_data
Enter year population borough: 2020 1390450 Bronx
Appended data to file: year: 2020	boro: Bronx	pop: 1390450

-update_data

Only showing the last few lines of output, but all the data should be displayed.

108 and 9999 4000 Stuy where entered into stdin.

        $ ./structrw -update_data
...
...MORE LINES HERE...
...
107: year: 2000	boro: Queens	pop: 2229379
108: year: 2000	boro: Bronx	pop: 1332650
109: year: 2000	boro: Staten Island	pop: 443728
110: year: 2010	boro: Manhattan	pop: 1585873
111: year: 2010	boro: Brooklyn	pop: 2504700
112: year: 2010	boro: Queens	pop: 2230722
113: year: 2010	boro: Bronx	pop: 1385108
114: year: 2010	boro: Staten Island	pop: 468730
115: year: 2020	boro: Bronx	pop: 1390450
entry to update: 108
Enter year pop boro: 9999 4000 Stuy
File updated.

How to deal with input

The above sections mentioned: The data should be entered to stdin on one line in the format: Year population borough

This can be done using sscanf() and some special notation for strings.

using %[SET] in your sscanf will scan a strings containing characters in the set until it sees somthing not in the set.

Using the ^ character will negate the set.

Example:

Parsing stdin strings using sscanf

char line_buff[256];
char s0[32];
char s1[32];
char s2[32];
char s3[32];
printf("Enter string: (I only read up to the first whitespace)");
printf("e.g. You cannot store 'abc def ghi'");
fgets(line_buff,255,stdin);
//read a string up to the first whitespace
sscanf(line_buff,"%s",s0);
printf("'%s'\n",s0);

printf("Enter string: (try 123abc)");
fgets(line_buff,255,stdin);
//read a string of digits, up to the first non-digit, start by consuming the newline
sscanf(line_buff,"\n%[0-9]",s1);
printf("'%s'\n",s1);

printf("Enter string: (try asdf123)");
fgets(line_buff,255,stdin);
//read a string of non-digits, stopping at the first digit.
sscanf(line_buff,"%[^0-9]",s2);
printf("'%s'\n",s2);

printf("Enter string: (try abc def ghi");
fgets(line_buff,255,stdin);
//read a string of non-digits, stopping at the first newline
sscanf(line_buff,"%[^\n]",s3);
printf("'%s'\n",s3);

[back to top of page]

2024-11-15

2024-11-15T12:00:00+00:00

2024-11-15

Processes

Every running program is a process.

A process can create subprocesses, but these are no different from regular processes.

A processor can handle 1 process per cycle (per core).

"Multitasking" appears to happen because the processor switches between all the active processes quickly.

On *nix style machines, processes are also files, listed in the /proc directory.

pid

Every process has a unique identifier called the pid.

pid 1 is the init process

The ps command shows status of processes. You will probably want the flags -aux, where -a will include other users, -x will include processes that are not attached to a terminal session (daemons and services), and -u shows more information including the username that ran the process.

ps -u or for even more processes ps -aux

There are many options, but they aren't super critical.

ps -u --sort pid

ps -u --sort %mem

Signals

Limited way of sending information to a process.

Sending a signal just sends an integer value to a process, these integers have special values like SIGINT (interupt signal) or SIGTSTP (keyboard signal to background a process).

Some keyboard shortcuts send signals to the active program in your shell:

ctrl-c will send the signal SIGINT.

ctrl-\ will send the signal SIGQUIT.

ctrl-z will send the signal SIGTSTP.

Sending signals via the terminal

kill - Command line utility to send a signal to a process

kill pid - Sends signal 15 (SIGTERM) to pid

kill -signal pid Sends signal to pid

$ killall [-signal_name] process_name - Sends SIGTERM (or signal_name if provided) to all processes with process_name

man 7 signal : shows all signals. 3: quit, 9: kill, 17/18: stop, 19: continue

Signals in your program

All these functions can be found in &ltsignal.h>

Finding PID:

getpid() - &ltunistd.h> - returns the current process- pid

getppid() - &ltunistd.h> - returns the current process- parent pid

Sending a signal:

kill(pid, signal)

Returns 0 on success or -1 (errno) on failure.
Works like the command line kill program

Listening for signals:

sighandler

To intercept signals in a c program you must create a signal handling function.
Some signals (like SIGKILL, SIGSTOP) cannot be caught.
static void sighandler( int signo )
- Must be static, must be void, must take a single int parameter.
- Static values in c exist outside the normal call stack, they can be accessed regardless of the function at the top.
- For variables, static also means they retain their value even if the function they are declared in has ended.
- Static values (variables and functions) can only be accessed from within the file they are declared.

signal

Attach a signal to a signal handling function
signal( SIGNUMBER, sighandler)
Note that you are passing the name of the signal handling function as a parameter.

singal/sighandler example:

Make the sighandler function:

static void sighandler(int signo) {
    if ( signo == SIGUSR1 )
      printf("Who you talkin to?\n”);
    }
  }

Then in your make sure you activate the sighandler by attaching it with the signal command:

signal(SIGUSR1, sighandler);

[back to top of page]

2024-11-18

2024-11-18T12:00:00+00:00

2024-11-18

static

static variables

preserve their value even after they are out of their scope! Hence, static variables preserve their previous value in their previous scope and are not initialized again in the new scope.
can only be initialized using constant literals. No function calls.
should not be declared inside a struct.
are initialized as 0 if not initialized explicitly.

static functions

Functions are declared static to limit access. They cannot be used in other files.
You can reuse of the same function name in other files as a result.

Example of a functin with a static variable:

#include 
int fun(){
    //This assignment only happens the first time the function is run
    static int count = 0;
    count++;
    return count;
}

int main(){
    printf("%d ", fun());
    printf("%d ", fun());
    printf("%d ", fun());
    return 0;
}

Output: 1 2 3

Lab10

Deadline : Thu 11/21 8am

You will have class time Today and Monday to work on it.

GitHub link: 10-Signals-That-You-Need-To-Sleep-More

Write a C program with the following features:

Runs constantly inside a forever while loop.
Print out the process' PID inside the loop "My PID is : XXX".
Use the sleep(1) function to have the program wait 1 second between inner loop iterations.
Catch the following signals
- SIGINT
  - Print to stdout "Process with PID: XXX Exiting due to SIGINT" replacing XXX with the actual process id.
  - Append the above message to a file output.txt, one message per line. The file should be created if it doesn't exist, or append new lines if it does. (See the notes on sprintf to format a string in memory like you do with printf).
  - Exit your program with a 0 exit code.
- SIGQUIT
  - Print out the PID and Parent PID in the format: "PPID: XXX , PID: YYY" replacing XXX and YYY with the correct ID values.
  - DO NOT cause the program to exit.
Include a makefile with a run and compile target.
I will make compile and then make run. I will then use the kill command (or ctrl-c and ctrl-\) to send the signals to your program.

Sample run:

$make compile
$make run
PID is : 10432
PID is : 10432
PID is : 10432
^-\ (sigquit)
PPID: 10401 , PID: 10432
PID is : 10432
PID is : 10432
^-c (sigint)
Process with PID: 10432 Exiting due to SIGINT
$cat output.txt
Process with PID: 10432 Exiting due to SIGINT
$

Please test your programs on a clean clone of your repo. The number of mistakes due to this last time was astoundingly high.

sprintf - print to a string buffer. This is used to format a string as if you were printing it. It is useful if you want to print the string, and do something else with it (like write it to a file also).

int sprintf(char *str, const char *string,...);

Return value:

If successful, it returns the total number of characters written excluding null-character appended in the string.

In case of failure a negative number is returned .

char buffer[100];
float pi = 3.14159;
int bytes = sprintf(buffer, "The value of pi is %.2f.", pi);
//check your bytes for negative
printf("%s\n", buffer);

2024-11-20

2024-11-20T12:00:00+00:00

2024-11-20

Exec

The exec functions are a group of c functions that can be used to run other programs.

When used, they replace the current process with the new program.

These functions are all in &ltunistd.h>

execl
execl(path, command, arg0, arg1 … NULL)
- path: The path to the program (ex: "/bin/ls" )
- command: The name of the program (ex: "ls")
- arg0 ... : Each command line argument you wish to give the program. (ex "-a", "-l") The last argument must be NULL
execlp
execlp(path, command, arg0, arg1 ... NULL)
- Works like execl, except it uses the $PATH environment variable for commands.
- For example, you can use "ls" as the path instead of "/bin/ls"
- To check the $PATH environment variable, use: echo $PATH
execvp
execvp(path, argument_array)
argument_array
- Array of strings containing the arguments to the command.
- argument_array[0] must be the name of the program.
- The last entry must be NULL
- Like execlp, the path argument will use the $PATH environment variable.
Sample argument_array

A much better example is displayed in class... your notes are supreme in this case.

How would you convert a command line into an argument array?

The way execvp works is it requires you to use an argument array. This means to run a command like "ls -a -l" must be parsed and manipulated into an array of 3 strings and a terminating null:
- argumentarray[0] must be "ls"
- argumentarray[1] must be "-a"
- argumentarray[2] must be "-l"
- argumentarray[3] must be NULL
Here is an example of what you would do in memory:

Note this is also why you must not destroy your buffer until after the tokens are no longer needed. Some of you had bugs where you used a buffer for your csv header, then re-used that buffer for the individual lines. At the end of your program, the header strings were erased.

This looks like a job for strsep()!

Remember you have a pointer to the string, that is mutable, in this case it is called string.

Store the locations of the different string starting points in your command args array.

After a few iterations:

[back to top of page]

2024-11-21

2024-11-21T12:00:00+00:00

2024-11-21

Lab11

Deadline: Tues 11-26

Main Goal

Write a function that uses strsep to parse a line of arguments

The library is parse.c/h your code will be tested by including parse.h so there should be no other extrernal requirements.
header: void parse_args( char * line, char ** arg_ary );
Parameter line a mutable string with the command line invocation of a program e.g. "ls -a -l" or "cat fish.txt"
Parameter arg_ary is an array of char* which will be modified to have each index point to the corresponding argument in the parameter string, that could be used for execvp()
Note that there is exactly one space separting adjacent arguments

Example:

char * cmdargv[16];
char line[] = "mkdir aaa";
parse_args(line,cmdargv);

Note the result of running the code above is the following array:

Also note that cmdargv is pointing to memory locations of line, so further manipulation of the line variable will result in damage to your array's strings.

Testing

To test this function, your main program (separate c file!) should use it on a string and then call execvp successfully.
For example, the following code should work assuming all variables are declared appropriately:
```
char * args[16];
char line[] = "ls -l -h";
parse_args( line, args );
execvp(args[0], args);
```
You can make the following assumptions:
1. There is exactly 1 space between each argument in the original string
2. char** arg_ary is long enough to store the tokens of the provided line
3. char* line is correctly formatted.
4. char* line is a muteable string

Assignment Repo:

Clone the repo: https://classroom.github.com/a/8ifXTxo7

Test with a variety of lengths of commands such as:

yes a b c d e f g h i (test with different numbers of args, but it must fit in your arg array)
echo 8 6 7 5 3 0 9 (test with different numbers of args, but it must fit in your arg array)
toilet -f emboss Execute! (you may not have this)
toilet Hello --metal (you may not have this)
cowsay -f dragon-and-cow roar

[back to top of page]

2024-11-25

2024-11-25T12:00:00+00:00

2024-11-25

Managing Sub-Processes

fork() - <unistd.h>

Creates a separate process based on the current one, the new process is called a child, the original is the parent.
The child process is a duplicate of the parent process.
All parts of the parent process are copied, including stack and heap memory, and the file table.
Returns a pid_t type (signed int), the value returned is:
- -1 if the fork failed, check your errno
- The child's pid to the parent
- 0 to the child
If a parent process ends before the child, the child's new parent pid is 1

pid_t p;
p = fork();
if(p<0){
  perror("fork fail");//output to stderr instead of stdout
  exit(1);
} else if ( p == 0){
    printf("Hello from Child!\n");
}else{
    printf("Hello from Parent!\n");
}

wait - <sys/wait.h>

Stops a parent process from running until any child has exited.
Returns the pid of the child that exited, or -1 (errno), and gathers information about the child process (this is called reaping)
If multiple child processes exit, an arbitrary one will be reaped.
wait(int* status)
- Pass in an address to an int status which is used to store information about how the process exited.
- Status macros
  - Usage: MACRO( status )
  - WIFEXITED: True if child exited normally
  - WEXITSTATUS: The return value of the child
  - WIFSIGNALED: True if child exited due to a signal
  - WTERMSIG: The signal number intercepted by the child

waitpid - <sys/wait.h>

Wait for a specific child
waitpid(pid, status, options)
- pid
  - The pid of the specific child to look for
  - If -1, will wait for any child (normal wait)
options
- Can set other behavior for waitpid,.
- This is 0 or more flags.
- if 0, will work normally, otherwise you would or the flags together.

Lab12

Repo link: https://classroom.github.com/a/0glxvVjC

Make should have compile and run targets.

Write a c program that does the following:

Forks off 2 child processes
Child process instructions:
- At the start print its pid, and a random number in the range [1, 5]. e.g. "123531 5sec". These numbers should be different each time, but could match sometimes (1/5 of the time NOT ALL THE TIME!)
- Sleep for a random number of seconds equal to the random number printed earlier (do not re-random the value!)
- Each child calculates their own random, so they can each sleep for a different number of seconds
- At the end print a message "PID finished" e.g. "123531 finished after 5 seconds."
Parent process instructions. Note the strings have _placeholders_ designated with underscores.
- Before forking, print out the initial message "_PID_ about to create 2 child processes."
- After the fork, wait for any child to finish
- Once a child has finished, print out a message. That includes its own PID and information about the child that exited. Then end the program.
- The message should be: "Main Process _PID_ is done. Child _EXITING_CHILD_PID_ slept for _SEC_ sec"
- The other child might still be running and can finish after the parent, that's intended behavior.

Sample output:

user@host $ ./a.out
68724 about to create 2 child processes
68726 2sec
68725 4sec
68726 finished after 2sec
Main Process 68724 is done. Child 68726 slept for 2sec
user@host $
68725 finished after 4sec

Note the 2nd child prints after your shell prompt is printed.

[back to top of page]

2024-11-27

2024-11-27T12:00:00+00:00

2024-11-27

Open a remote repo in a browser using the CLI:

WSL:

alias gh='explorer.exe https://$(git config remote.origin.url | cut -f2 -d@ | tr ':' /)'

Non-wsl:

Change the explorer.exe to some other application... like firefox , or on mac, you can use the open command

Like this: firefox "https://`(git config remote.origin.url | cut -f2 -d@ | tr ':' /)`"

SHELL PROJECT After break:

You (and up to 2 partners if desired - at least 1 partner highly suggested) are to write a shell.

By Monday, you should have a group chosen. You may not work with the same partner as before.

Redirection

Redirection is when you change the usual input/output behavior of a program

Command line redirection

Symbol	Purpose
&gt	Redirects stdout to a file. Overwrites the contents of the file.
&gt&gt	Redirects stdout to a file by appending.
&lt	Redirect stdin from a file. The file is treated exactly like stdin, for example scanf() will read up until a newline is found.
\| (pipe)	Redirect stdout from one program to stdin of the next. Very useful for chaining programs together.

Redirection in c programs

dup2 - &ltunistd.h> is used to redirect a file descriptor
- dup2( fd1, fd2 )
- Redirects fd2 to fd1
- Any use of fd2 will now act on the file for fd1.
dup - &ltunistd.h> is used to backup a file descriptor.
- Duplicates an existing entry in the file table.
- Returns a new file descriptor for the duplicate entry.
- dup( fd )

USING dup and dup2 together:

Note: the filenumber for stdout is just stdout

      int fd1 = open("foo.txt", O_WRONLY);
int stdout = STDOUT_FILENO;//stdout filenumber is 1, but this makes it clear
int backup_stdout = dup( stdout ) // save for later
dup2(fd1, stdout); //sets stdout's entry to the file "foo.txt".
printf("TO THE FILE!!!\n");
fflush(stdout);//not needed when a child process exits, becaue exiting a process will flush automatically.
dup2(backup_stdout, stdout); //sets the stdout entry to backup_stdout, which is the original stdout

Your shell project will need to use dup() and dup2()

It should be reasonably straight forward that you can implement the redirection operators &lt or &gt by changing the input to be a file instead of stdin, or the output to be a file not stdout.

Implementation

You can make redirection with pipes ls | wc in the following manner:

run ls and redirect stdout to a temp file
run wc and redirect the temp file to stdin
remove the temp file when done.

The above strategy is a little inelegant, as it would pollute the ls command with the temp file. There is a better way we will learn in the near future.

[back to top of page]

2024-12-02

2024-12-02T12:00:00+00:00

2024-12-02

Cowabunga, let's kick some shell 🐢

Project (MyShell) Overview

You (and up to 2 partners if desired - at least 1 partner highly suggested) are to write a shell. The shell should implement as many of these features as possible. The first being the most basic up to the last which is the most complex.

General Functionality:
1. Prompt the user for input and read a line at a time from stdin.
2. parse the user input line to separate the command from its arguments.
3. The shell should then fork for each command and the child should execvp that command.
4. The parent process (shell) should wait until the exec'd program exits and then it should go back to step 1.
5. Semicolons act as a separator of commands, so you can have multiple commands on the same line. e.g. echo 1;echo 2;echo 3 Important note: The semicolon is the only symbol that will not be separated by spaces (for easy splitting)
The working directory:
1. The command cd is not a program to execvp, your shell has to keep track of the current working directory instead.
2. Look at the chdir() and getcwd() functions
The prompt:
1. The prompt should includ the cwd path, e.g. "/home/konstans/Documents/ $"
2. The prompt can shorten the path using the ~ as the home directory. e.g. "~/Documents/ $"
3. You should fflush(stdout); after printing so that a partial lines are displayed and the children don't print a second copy of things. (The shell prompt is a partial line!)
Exiting:
1. To quit your shell use the exit command. This is not a program to execute.
2. Ctrl-D (end of file) shoud terminate the shell. Check your fgets() return value.
Implement simple redirection.
1. Support both redrection oprators: the &gt (redirecting stdout) and &lt (redirecting stdin).
2. This will be explained in class.
Implement simple pipes. Only one pipe at a time is required.
1. The pipe operator | redirects one program's output to be another program's input. e.g. ls | wc would run ls and use the output from ls as the input for wc.
2. Check out popen() for the advanced way of doing this, otherwise use a temp file.
Compilation, running, and testing:
1. The project should be built using make and a makefile, I will only compile by using make compile which should output an executable file shell .
2. The project should be runnable by running ./shell after the make compile command.
3. You must be able to test your shell using the syntax: ./shell &lt lines.txt where lines.txt contains commands on each line.
4. All of the features discussed work in bash, so use it to check out the functionality of each feature if you are unclear about anything.

Code/Documentation requirements

Every function you write must have a function header comment describing the arguments, return value and what the function does.
Use modular design liberally. This is a large program.
You should have a readme file with the following:
- All group member's names
- Your group's "creative team name" where you can optionally flex on other groups by making a really good name.
- A description of what features your shell implements
- A description of what features you attempted to implement but were unsuccessful
- Any bugs or things you want me to know about your program
- I am much more forgiving of reported bugs than bugs I discover while testing
- A copy of every function header
This project should use separate C files as necessary.
Restrictions on input (to make your job easier):
- Command line arguments will be separated by single spaces. (Except the semicolon)
- I will only test a single pipe |
- I will only test up to one &gt
- I will only use the &gt syntax, not the 2&gt or &amp&gt
- &lt would go after the first command only,
- &gt would go at the end of the command only.
- e.g.
```
a &lt c.txt | b  &gt d.txt
```
  This would cause: c.txt goes into program a, the output of a goes into program b, the output of b goes into the file d.txt )
If you would like to implement other features after getting these ones down, please do. Look at what bash does and see what else you can do!

Due: Monday 12/09 8:00 am

Repo Name format:

PERIOD-NAME1-NAME2 (This is separate from your creative team name)

Where NAME is last name + first initial e.g. 3-WrightE-WongL-WongJ

Team name and other information will go in the README.md

GitHub Submission Link:

https://classroom.github.com/a/Tfg6waJb

[back to top of page]

2024-12-09

2024-12-09T12:00:00+00:00

2024-12-09

Inter-Process Communication (IPC)

There are many kinds of Inter-Process Communication, but there is a specific set of IPC features that have competing standardized implementations, these are:

Shared Memory
Semaphores
Message Queues

System V IPC

For the IPC features mentioned, there are 2 standards:
1. Portable Operating System Interface (POSIX)
2. System V
Both of these cover more than IPC, and their functionality, thought not their implementations, are mostly the same.
We will be using System V IPC.
$ ipcs is a useful command line utility to see active System V IPC structures.

Shared Memory

A segment of heap-like memory that can be accessed by multiple processes.
Shared memory is accessed via a key that is known by any process that needs to access it.
Shared memory does not get released when a program exits.
5 Shared memory operations
- Create the segment (happens once) - shmget
- Access the segment (happens once per process) - shmget
- Attach the segment to a variable (once per process) - shmat
- Detach the segment from a variable (once per process) - shmdt
- Remove the segment (happens once) - shmctl

Using shared memory in C: Headers: <sys/shm.h> <sys/ipc.h> <sys/types.h>

int *data;
int shmid;
shmid = shmget(INTEGER_KEY_GOES_HERE, sizeof(int), IPC_CREAT | 0640);
printf("shmid: %d\n", shmid);
data = shmat(shmid, 0, 0); //attach
printf("data: %p\n", data);
printf("*data: %d\n", *data);
*data = * data + 10; //work with the segment as a normal pointer
printf("*data: %d\n", *data);
shmdt(data); //detach

To remove the segment:

int shmid;
shmid = shmget(INTEGER_KEY_GOES_HERE, sizeof(int), IPC_CREAT | 0640);
shmctl(shmid, IPC_RMID, 0); //remove the segment

Controlling access

Before utilizing shared memory, we need a way to control access to shared resources. Files and shared memory are both resources that are commonly shared.

You have seen lockfiles, or files that tell you another file is being edited by some other program. These are common for files specifically, but they don't always work. When programs crash, they do not clean up the lock files. If you ever had a "firefox is already running error" or seen a file in your directory with almost the same name as something you are editing, that is an example of a file that controls access.

Unfortunately files to control access are not appropriate for shared memory.

Tomorrow we will learn about semaphores. A semaphore is method of visual signaling, usually by means of flags or lights.

Monty Python semaphore video

Jump to the 18 minute mark of this video and watch the semaphore version of withering heights: https://www.dailymotion.com/video/xsefeg

Now you will remember that a semaphore is method of visual signaling, usually by means of flags or lights.

In a programming language we use semaphores to "notify" a program that a resource is either free or in use much like how the flags signal.

Semaphores in c

Semaphores are an IPC construct used to control access to a shared resource (like a file or shared memory).

Most commonly, a semaphore is used as a counter representing how many processes can access a resource at a given time.

If a semaphore has a value of 3, then it can have 3 active “users”.
If a semaphore has a value of 0, then it is unavailable.

Some semaphore operations are atomic, meaning they will not be split up into multiple processor instructions.

Semaphore operations

“Maintenance operations”
- Create a semaphore
- Set an initial value
- Remove a semaphore
Traditional Semaphore Usage
- Up(S) | V(S) - atomic
  - Release the semaphore to signal you are done with its associated resource
  - Pseudocode
    - S++
- Down(S) | P(S) - atomic
  - Attempt to take the semaphore.
  - If the semaphore is 0, wait for it to be available.
  - Pseudocode
    - while (S == 0) { block } S--;
  - Important distinction that the blocking is not atmoic, so if a processess checks the semaphore and it is unavailable, other processes can run, but if the semaphore is available, the process will immediately modify the semaphore.

[back to top of page]

2024-12-10

2024-12-10T12:00:00+00:00

2024-12-10

Using semaphores in C

headers: <sys/types.h> <sys/ipc.h> <sys/sem.h>
semget
- Create/Get access to a semaphore.
- Returns a semaphore descriptor or -1 (errno)
- semget( key, amount, flags )
  - key
    - Unique semaphore identifier
  - amount
    - Semaphores are stored as sets of one or more. The number of semaphores to create/get in the set.
  - flags
    - IPC_CREAT: create the semaphore and set value to 0
    - IPC_EXCL: Fail if the semaphore already exists and IPC_CREAT is on.
    - Includes permissions for the semaphore, combine with bitwise or (|).
semctl
- Control the semaphore, including
  - Set the semaphore value
  - Remove the semaphore
  - Get the current value
  - Get/set semaphore metadata
- semctl(descriptor, index, operation, data)
  - descriptor
    - The return value of semget
  - index
    - The index of the semaphore you want to control in the semaphore set.
  - operation
    - IPC_RMID: remove the semaphore
    - SETVAL: Set the value (requires data)
    - GETVAL: Return the value
- data
  - Variable for setting/storing semaphore metadata
  - Type is union semun
  - You have to declare this union in your main c file on linux machines.
    - ```
    union semun {
      int val;                  //used for SETVAL
      struct semid_ds *buf;     //used for IPC_STAT and IPC_SET
      unsigned short  *array;   //used for SETALL
      struct seminfo  *__buf;
    };
                        
```
- union?
  - A c structure designed to hold only one value at a time from a group of potential values.
  - Just large enough to hold the largest piece of data it could potentially contain
semop
- Perform an atomic semaphore operation
- You can Up/Down a semaphore by any integer value, not just 1
- semop( descriptor, operation, amount )
  - amount - The amount of operations you want to perform on the semaphore set.
  - operation - A pointer to a struct sembuf
- struct sembuff:
```
struct sembuf {
  short sem_op;
  short sem_num;
  short sem_flag;
};
```
  - sem_num
    - The index of the semaphore you want to work on.
  - sem_op
    - Down(S): Any negative number
    - Up(S): Any positive number
    - 0: Block until the semaphore reaches 0.
  - sem_flag
    - SEM_UNDO: Allow the OS to undo the given operation. Useful in the event that a program exits before it could release a semaphore.
    - IPC_NOWAIT: Instead of waiting for the semaphore to be available, return an err

Putting it all together.

Creating a single semaphore and initializing its value to 1:

int semd = semget(KEY, 1, IPC_CREAT | IPC_EXCL | 0644);
union semun us;
us.val = 1;
r = semctl(semd, 0, SETVAL, us);

Removing a semaphore: semctl(semd, IPC_RMID, 0);

Getting a a semaphore & upping and downing it.

semd = semget(KEY, 1, 0); //get access
struct sembuf sb;
sb.sem_num = 0;
sb.sem_flg = SEM_UNDO;
sb.sem_op = -1; //setting the operation to down

semop(semd, &sb, 1); //perform the operation
printf("got the semaphore!\n");
sleep(10); //simulate doing something.

sb.sem_op = 1; //set the operation to up
semop(semd, &sb, 1); //perform the operation

Sample IPC code:

sem_control
sem_user
shm
shm2
shmrm
unnamed(for pipes later in the week)

Lab13

Deadline Thursday, you will have 2 lab days to work in class.

https://classroom.github.com/a/lZQ6Uun6

You're going to write a game where each program adds on a new line to a story, but they can only see the one previous line!

It will use a semaphore to control access to a file and shared memory.

Before doing any of this check out the notes page for _information on using semaphores in c. At the bottom are full code snippets that demonstrate the usage. I will add some sample programs that use shared memory and semaphores.

You will need 2 programs, one to create/monitor/remove the file (stores a story), shared memory (stores the size of the last line added.) and a semaphore (to block write access), and one to use those structures to implement the telephone style game:

Both programs should be compiled with the compile target of your makefile.

Note: Only one program should be able to modify the resources at a given time. This includes creating and writing, but not reading!

Control program:

The control program should be run with a make target "control" (make control ARGS=??? e.g. make control ARGS=create)

create

Should make the shared memory segment (one integer), semaphore and file called story.txt (open the file with the truncate flag to erase any prior data.)
Set any values that are needed.

remove

Remove the shared memory and the semaphore
Display the full contents of the story.

view

Display the full contents of the story.

The writing program:

Your program will allow multiple people to collaborate on a story, BUT the intent is that you can only see one previous entry when adding new text.

Should be run with a make target write. ( make write )
Will attempt to access the resources using the semaphore.
Display a message indicating that it attempting to open the resource. This means there will be some message prior to the program blocking.
Once it gains access to the resources it should:

Finds the size of the last line added by looking at the value in the shared memory. If the size is 0, then you are the first person.
Display the last line added to the file (seek using the size).
Prompt the use for the next line to be added to the story.
Read the user input from stdin
Once a new line of input is read, it should write that message to the file and write the size to the shared memory .
After completing these steps you release the semaphore, and end the write program.

[back to top of page]

2024-12-13

2024-12-13T12:00:00+00:00

2024-12-13

Pipes

A pipe is a conduit in memory between 2 separate processes on the same computer.

Pipes have 2 ends, a read end and a write end. Pipes act just like files (i.e. you can use read() and write() to send any set of bytes).

Unnamed pipes have no external identifier (more on named pipes later).

Working with pipes

Pipes are in &ltunistd.h>

pipe( descriptor_array ) creates an unnamed pipe.

Returns 0 if the pipe was created, -1 if not (errno).

This opens two file descriptors, the reading end and the writing end.

descriptor_array will contain the descriptors for each end of the pipe. Must be an int array of size 2.

descriptor_array[0] is the read end.

descriptor_array[1] is the write end.

Example:

//it is usefull to add these definitions to make your code more readible
#define READ 0
#define WRITE 1
int main() {
  int fds[2];
  pipe( fds );
  char line[100];

  f = fork();
  if (f == 0) {
    close( fds[READ] ); //it is a good idea to close the end of the pipe your are not using.
    write( fds[WRITE], "hello!", 7);
  }
  else {
    close( fds[WRITE] );
    read( fds[READ], line, sizeof(line) );
  }
}

Named Pipes

Named pipes are also known as FIFOs.

Same as unnamed pipes except FIFOs have a name that can be used to identify them via different programs.

Like unnamed pipes, FIFOS are unidirectional.

mkfifo is a shell command to make a FIFO

$ mkfifo nameOfPipeHere

mkfifo is a c function to create a FIFO that requires: &ltsys/types.h> &ltsys/stat.h>

Returns 0 on success and -1 on failure
Once created, the FIFO acts like a regular file, and we can use open , read, write, and close on it.
FIFOs will block on open until both ends of the pipe have a connection.

Classwork + Homework:

Design a program that uses unnamed pipes to send one-way messages from a parent to a child or vice versa.

The parent can prompt the user for a message, and the child process can print it.

You should print the pid of the process in your printf statements so you know which process is saying what. e.g.

$ make run
pid 32041: Enter text: blah blah backslash.
pid 32050: Recieved message: blah blah backslash.
$

[back to top of page]

2024-12-16

2024-12-16T12:00:00+00:00

2024-12-16

Lab 14

Deadline: 12-18 8am

https://classroom.github.com/a/BrV3rDjy

Phase 1

Design a program that uses unnamed pipes to facilitate two-way parent/child process communication.

Hint: In order to facilitate 2 way communication, you will need 2 pipes.

Parent
1. constantly prompts a user for input
2. sends that input to child
3. waits for a response
4. displays the response to the user
Child
1. Get input from parent
2. "processes" the input. process could be any of a number of string processes (all caps, all lower, reverse, rot13, etc. ).
3. Send the processsed string back to the parent.

Phase 2

Add the following features to your lab:

Use one named pipe (create in the same directory) for child processes to communicate back with the parent.
Instead of 1 child, spawn 2-4 child processes randomly, give each their own unnammed pipe to receive messages from the parent.
You may wish to create a "childID" variable that helps you know which pipe to use
The parent will send all 4 processes the same input text.
Each process will sleep a random amount of time 0-4 seconds. To do this, you should use the system call usleep() which uses microseconds instead of seconds, so random int from 0 to 4000000.
After sleeping the child processes will process the string and send the pid of the child process + the processed string back to the parent using the named pipe.
You should use one write command to send all the data to avoid collisions with the other child processes.
The parent will then print the PID and the processed string for each child
The parent loops and asks for more input, use ctrl-c to exit.
Note: There is no reason to use a named pipe here except to learn to used named pipes.

Output format to make y'all have some consistency

$make run
Spawning 4 worker threads.
User Control Program (1244909) enter a line:
one fish two fish
(1244910) sleeping for 2.565000 seconds
(1244913) sleeping for 3.185000 seconds
(1244911) sleeping for 0.066000 seconds
(1244912) sleeping for 3.802000 seconds
result 0: pid(1244911): tsj knxm ybt knxm
result 1: pid(1244910): sri jmwl xas jmwl
result 2: pid(1244913): vul mpzo adv mpzo
result 3: pid(1244912): utk loyn zcu loyn
User Control Program (1244909) enter a line:
^C
$make run
Spawning 2 worker threads.
User Control Program (1244993) enter a line:
RED fisH! BluE fash.
(1244994) sleeping for 3.856000 seconds
(1244995) sleeping for 0.436000 seconds
result 0: pid(1244995): CPO qtdS! MwfP qlds.
result 1: pid(1244994): BON pscR! LveO pkcr.
User Control Program (1244993) enter a line:
fish fish
(1245066) sleeping for 1.327000 seconds
(1245067) sleeping for 1.374000 seconds
result 0: pid(1245066): jmwl jmwl
result 1: pid(1245067): knxm knxm
User Control Program (12444993) enter a line:
^C

[back to top of page]

2024-12-18

2024-12-18T12:00:00+00:00

2024-12-18

Pipe Networking

For the purposes of these notes, the words server and client will be used only to differentiate two programs with respect to their roles in a connection attempt, and not with respect to their usage once a connection is made.

Server will be the program that starts up and awaits an incoming connection.
Client will be the program that initiates a connection to a waiting server.

Handshake

A handshake procedure to ensure that a connection has been established between 2 programs.

Both ends of the connection must verify that they can send and receive data to and from each other.

3 way handshake

Client sends a message to the server (in TCP networking, this is called SYN). At this point, the server knows it can receive data.
Server sends a response to the client based on the client's initial message (SYN_ACK). At this point the client knows it can recieve and send data.
Client sends a response back to the server based on the server's response (ACK). At this point, the server knows it can receive and send data.

Server-Client Designs

There are a number of different ways to implement server/client systems.

Single Use Server

In this system, the server exits along with the client.

Handshake
Client: sends data to server.
Server: gets response, acknowledges the response.
Client: gets the response, acknowledges the acknowledgement.

When the client exits, so does the server.

Persistent Single Client Server

Pay attention to the setup/handshake which will be useful for your lab.

In this system, the server will communicate with a single client. When the client exits, the server will reset to handle a new client.

A Well Known Pipe (WKP) is a named pipe that exists only while the server exists, all clients know the name.

A Private Pipe (PP) is a pipe that exists while the client is running. The client tells the server about this pipe. We will use a named pipe with the name that is the PID of the client.

Setup
1. Server creates a WKP and waits for a connection.
2. Client creates a private fifo (can use the PID in the name so it is unique.)
3. Server waits for a connection to the WKP.
Handshake
1. Client: sends a message to server. (SYN) We will send the PP name to server. e.g. "120350"
2. Server: gets the message, Removes the WKP. Sends an (SYN_ACK) We can send a random int to the PP.
3. Client: gets the acknowledgement. Removes the PP. Sends the (ACK) its own acknowledgement to the server (same random integer + 1).
4. Server reads and verifies the 2nd acknowledgement. This completes the handshake and both ends know that both send/recieve are working.
Operation
1. Server and client send data back and forth in a pre-determined pattern (protocol).
Reset
1. Client exits, server closes any connections to the client.
2. Server goes back to setup and recreates a WKP...

The only way to quit the server is via ctl-c. This is fine, but it will leave the WKP on the filesystem. A cleaner exit would involve creating a signal handler that catches SIGINT and remove the WKP before exiting. Some of you did this on your prior lab.

Forking Server

Pay attention to the setup/handshake which will be useful for your lab.

In this system, the server will create subservers for client communication. This will allow for multiple simultanious connections.

The main server's job is to wait for a connection and create subservers.

The subserver will handle all communication with the client.

The client can work exactly the same as the persistent server client.

Server: creates WKP and blocks until connection.
Client: creates PP, connects to WKP, sends PP name, blocks on connection to PP.
Server: gets connection on WKP, creates subserver.
Server: Closes & removes WKP from disk.
Server: Resets back to step 1.
Subserver: reads PP name from client. (Subserver still has the WKP in memory)
Subserver: sends secret message to client.(ack1)
Client: gets subserver secret, removes PP, responds with (ack2).
Subserver: verfies response from client, completing 3-way handshake.

The following table will show the ordered sequence or steps. The steps can happen concurrently EXCEPT for when either server or client is blocked due to opening a pipe or reading.

This sequence also shows how to design a setup()/handshake() method for the server/client.

$ ./server	$ ./client
-- server_handshake() --	-- client_handshake() --
-- server_setup() --	3 Client making Private pipe
1 Server making the pipe.	3 Client opening WKP [Unblock Server]
2 Server opening the WKP.[BLOCKS]	3 Client Writing PP to WKP (matches with step 5)
4 Server removing the WKP.	3 Client Opening PP [BLOCKS]
-- end server_setup() --	8 Client Deleting PP
5 Server reading SYN (the pid)	8 Client reading SYN_ACK (matches with step 7)
6 Server opening the Private Pipe [Unblock client]	8 Client Sending (matches with step 9) ACK
7 Server sending SYN_ACK	-- end client_handshake() --
9 Server reading final ACK
9 Server received ACK, handshake complete
-- end server_handshake() --

Classwork/HW

Clone the lab: https://classroom.github.com/a/qfzEQx6Z

Fill in the function definitions for server_handshake and client_handshake in pipe_networking.c.
These should do everything needed to establish a 3-way handshake between the client and server.
Make sure you include diagnostic print messages in your functions.

Note: To remove files, there is a simple function remove(PATH) that will remove the file at PATH.
Make and test
Your final version should not modify any other files for this assignment.
You CAN test to see if the handshake worked by writing a byte from client to server and vice versa.

[back to top of page]

2025-01-02

2025-01-02T12:00:00+00:00

2025-01-02

Upgrade your pipe server!

Deadline is Tuesday.

You should have tested your client and server by sending information back and forth somehow.

Basic client/erver functionality:

Now we want our server to choose a random int x from 0 to 100 and then send it to the client. Do this once every second (sleep after sending).

Now we want our client (all clients will be this same behavior) to reapatedly: read the int sent by the server and print it.

If you ctrl-c the client or the server they should both exit gracefully.

Update 1

Make a new file for this part: persistant_server.c™ (copy your working basic_server.c and start with that).

Have the server loop to accept a new clients after the first client disconnects (ctrl-c the client). That is many sequential clients, one after another.
Close the old file descriptors before you end your loop, or you will have an ever-growing file table. (close from_client and to_client)
You must now also add a sighandler for SIGINT. This will delete the WKP befor ending the server.

Update 2

Make a new file for this part: forking_server.c (copy your working persistant_server.c and start with that.) This server fork to handle multiple concurrent connections.

The server should only:

create the WKP
await a connection
upon connection remove the WKP.
Fork to create a subserver which will handle the client.
Go back to step 1.

A subserver should:

Complete the 3-way handshake.
Send random ints to the client repeatedly as before.
Close when the client exits.

Notice the server steps 1-3 are all contained in the server setup command! Your server sets up the handshake, but your subserver actually does the rest of the handshake.

You should write a half-handshake function that takes in the from_client fd as an additional argument: server_handshake_half(int *to_client, int from_client)

Your server will server_setup() and your sub-server will finish the job with server_handshake_half()

To provide some communication, after the handshake:

Repeatedly Send a random number to the cleint (use sleep(1), client prints the value.)

You can test by spawning multiple clients and verifying they all can maintain communication with the subservers

The end result should be one server that processes client input from any number of clients.

You can spawn 2 or more clients, each of which should be printing data every second.

UPDATE:

Required make targets:

compile as usual: make compile
to run the forking server: make server
to run the client: make client

I will run multiple clients in separate terminals this way.

Final Projects

Project requirements

The final project is to be a significant piece of work that illustrates many of the ideas and techniques studied in the class:
- Allocating memory
- Working with files
- Finding information about files (stat and such)
- Processes (fork, exec etc.)
- Signals
- Shared memory
- Semaphores
- Pipes (named and unnamed)
- Sockets
Your project should be able to meaningfully use at least 4 of these concepts. Don't just toss in a topic if there is no need for it in your project. Part of your design process should include considering how to incorporate the class material.
You should also focus on creating a program that works well. It is better to have fewer features in a project that compiles & runs smoothly than to have a lot of features that contain a lot of bugs.
When designing your project, it is best to be able to generate a base working project early and then add features modularly as time goes on.
You may work with one or two partners, all partners should be in the same section.

Project Ideas/Suggestions

If you are having difficulty coming up with a project idea, look back at some of the bigger assignments we-ve done over the semester and think about how you can add to them. These are just some ideas to get you going, please do not consider theses as superior to any other ideas you might have.
- The music library could become much more useful by incoperating a music player (fork/exec), saving the library data (maybe even as a binary file), interacting with actual music files etc.
- The data file/csv assignment could become a more useful data organizing program.
- Semaphone could be tunred into a pipe/socket based live game.
- With pipes/sockets, you could write simplified versions of different kinds of servers.
- With pipes/sockets and semaphores you could write a SIMPLE database program.
If you are going to write a multi-program system with sockets/pipes, start simply. For example, if you-re making a 2 player game, then just have the programs connect directly do each other, instead of wirting a separate server to connnect them.
As stated earlier, it's better to start small and grow than start large and have to cut things out (or live with bugs).

URGENT NOTE:

we have not covered networking and we are about 1 week short to cover the material.

I will post notes, and sample code, but do NOT suggest networking except for groups that want to learn more on their own.

Prior Projects

Distributed computing tool
file editor (text/spreadsheet)
Database program
Client server program
- chat
- simple game many players concurrently
- 2-player game tournament

[back to top of page]

2024-12-18

2024-12-18T12:00:00+00:00

2024-12-18

Motivation: Wouldn't it be nice if your server could accept clients from different computers? You could then distribute a client to your classmates and have them all use it.

Sockets

A connection between 2 programs using network protocols.

This is usually between 2 computers, but does not have to be.

A socket corresponds to an IP (internet protocol) Address / Port pair.

To use a socket:

create the socket: `socket`

bind it to an address and port: `bind`

listen & accept/initiate a connection: `listen` `accept`, `connect`

send/receive data

Functions vary depeding on type of socket

Most functions and structres in `sys/socket.h`

`socket( domain, type, protocol )`

Creates a socket, opens it like a file, returning a socket descriptor (int that works like a file descriptor)

`domain`: type of address

`AF_INET` or `AF_INET6` or `AF_UNSPEC`

`type`

`SOCK_STREAM` or `SOCK_DGRAM`

`protocol`

Combination of domain and type settings

If set to 0 the OS will set to correct protocol (TCP or UDP)

example: `int sd = socket(AF_INET, SOCK_STREAM, 0);`

Socket Protocols

Stream Sockets (TCP)
- Reliable 2 way communication.
- Must be connected on both ends.
  - 3 way handshake
- Data is received in the order it is sent.
Datagram Sockets (UDP)
- “Connectionless”: an established connection is not required.
- Data sent may be received out of order (or not at all).
- Cannot use the usual read and write function calls.

Sockets in C

Most functions and structres in sys/socket.h
socket( domain, type, protocol )
- Creates a socket, opens it like a file, returning a socket descriptor (int that works like a file descriptor)
- domain: type of address
  - AF_INET or AF_INET6 or AF_UNSPEC
- type
  - SOCK_STREAM or SOCK_DGRAM
- protocol
  - Combination of domain and type settings
  - If set to 0 the OS will set to correct protocol (TCP or UDP)
- example: int sd = socket(AF_INET, SOCK_STREAM, 0);
getaddrinfo <sys/types.h> <sys/socket.h> <netdb.h>
- System library calls use a struct addrinfo to represent network addresses (containing information like IP address, port, protocol…)
- Will lookup information about the desired network address and get one or more matching struct addrinfo entries as a linked list.
- getaddrinfo(node, service, hints, results)
- node: String containing an IP address or hostname to lookup
  - If NULL, use the local machine's IP addresses (all of them).
- service: String with a port number or service name (if the service is in /etc/services)
- hints: Pointer to a struct addrinfo used to provide settings for the lookup.
  - Think of this as a filter for the lookup that getaddrinfo performs. For example, if you only want to get an IPv4 address, you can set hints to be an IPv4 address.
- results: Pointer to a linked list of struct addrinfo containing entries for each matching address.
- getaddrinfo will allocate memory for these structs. Since results will be a linked list of unknown size, you should use freeaddrinfo to release the entire linked list when you are done.
bind( socket descriptor, address, address_length) (server only)
- Binds the socket to an address and port. This allows a server has to have a set public* adrress and port.
- Returns 0 (success) or -1 (failure)
- socket descriptor: return value of socket
- address: pointer to a struct sockaddr representing the address.
- address_length: Size of the address, in bytes
- address and address_length can be retrieved from getaddrinfo.

Using getaddrinfo, socket & bind on a server

//use getaddrinfo
  struct addrinfo * hints, * results;//results is allocated in getaddrinfo
  hints = calloc(1,sizeof(struct addrinfo));
  hints->ai_family = AF_INET;
  hints->ai_socktype = SOCK_STREAM; //TCP socket
  hints->ai_flags = AI_PASSIVE; //only needed on server
  getaddrinfo(NULL, "9845", hints, &results);  //Server sets node to NULL

  //create socket
  int sd = socket(results->ai_family, results->ai_socktype, results->ai_protocol);

  bind(sd, results->ai_addr, results->ai_addrlen);

  //DO STUFF

  free(hints)
  freeaddrinfo(results);

listen (socket_descriptor, backlog) (server only)
- Set a socket to passively await a connection.
- Needed for stream sockets.
- Does not block.
- socket descriptor: return value of socket
- backlog: Number of connections that can be queued up.
  - Depending on the protocol, this may not do much.
accept (server only)
- Accept the next client in the queue of a socket in the listen state.
- Used for stream sockets.
- Performs the server side of the 3 way handshake
- Creates a new socket for communicating with the client, the listening socket is not modified.
- Returns a descriptor to the new socket
- Blocks until a connection attempt is made
- accept(socket_descriptor, address, address_length)
- socket descriptor: descriptor for the listening socket address: Pointer to a struct sockaddr_storage that will contain information about the new socket after accept succeeds.
- address length: Pointer to a variable that will contain the size of the new socket address after accept succeeds.

Using listen and accept for servers

//use getaddrinfo (not shown)
  //create socket
  int sd = socket(results->ai_family, results->ai_socktype, results->ai_protocol);
  //use bind
  bind(sd, results->ai_addr, results->ai_addrlen);
  listen(sd, 10);
  int client_socket;
  socklen_t sock_size;
  struct sockaddr_storage client_address;
  sock_size = sizeof(client_address);
  client_socket = accept(sd,(struct sockaddr *)&client_address, &sock_size);

Using connect for clients

connect (client only) <sys/socket.h> <sys/types.h>
- Connect to a socket currently in the listening state.
- Used for stream sockets.
- Performs the client side of the 3 way handshake
- Binds the socket to an address and port
- Blocks until a connection is made (or fails)
- connect(socket descriptor, address, address length)
- socket descriptor: descriptor for the socket
- address: Pointer to a struct sockaddr representing the address.
- address length: Size of the address, in bytes
- address and address length can be retrieved from getaddrinfo()
- Note that the arguments mirror those of bind()

//use getaddrinfo (not shown)
  //create socket
  int sd = socket(results->ai_family, results->ai_socktype, results->ai_protocol);

  connect(sd, results->ai_addr, results->ai_addrlen);

Beej's Guide to network programming using internet sockets

https://beej.us/guide/bgnet/html/

2024-12-19

2024-12-19T12:00:00+00:00

2024-12-19

Networking.

Layer Models of Networking

Due to the complexity of network communications, the topic is often conceptualized into distinct layers so people can work on specific components rather than everything at once.

The bottom layer is the most concrete, with each subsequent layer becoming more abstract (relying less on the physical connections and more on code).

There are various competing models, including the OSI (Open Systems Interconnections) and TCP/IP Models.

TCP/IP Model Layers

Application - (process)
Transport - (OS)
Network - (OS)
Link - (Physical)

Link Layer

Point-to-point transmission between devices on the same (local) network.

Combines physically connecting computers with basic addressing and transmission protocols.

Physical connection

How to transmit bits between two computers.
Electrons, photons, radio waves…

Link Layer: A brief history of physical connections

Thicknet

A single coaxial cable runs along the network.

“Vampire taps” cut into the cable and connect to a computer.

Thinnet

A single coaxial cable runs along the network.

T-Connectors connect computers to the main cable.

Thin/Thicknet network topology

Each new computer degrades the network collisions could cause issues

Token Ring

Each computer is connected in a ring to each other.

Only one computer has command of network resources at a time. This is called "having the token”.

The network sends a "token" throughout the ring, which contains the identity of the computer allowed to use the network. All other computers must wait to send data over the network.

Each computer provides its own power and keeps single strength constant.

No possibility of collisions

Ethernet

Multiple computers connect to a single hub or switch.

Hub: Broadcasts the data to all the computers

Switch: Sends data to a specific computer

[back to top of page]

2024-12-20

2024-12-20T12:00:00+00:00

Link Layer: The Softer Side

In order for data to be sent between computers:

Each computer needs a unique address (MAC Address).
The data needs to be sent in a standardized format (Frames).

MAC (Media Access Control) Address

6 - Byte Hex address: 2a:00:1e:b9:70:f6
MAC addresses only need to be unique on the same local network.

Link Layer: The Softer Side

Ethernet Frames

Each frame has the following format:

prefix dest source type data checksum

prefix 8B: 10101010 x7 + 10101011

dest & source 6B each: MAC addresses

type 2B

data: MTU (Maximum Transmission Unit) of 1500B

checksum 4B: ensures data integrity

prefix	dest

prefix	dest	source	type	data	checksum
8B	6B	6B	2B		4B

Internet Layer

Transmission of data between two separate networks.

Major features of this layer are addressing and routing.

Routers are physical devices used to connect different local networks.

Internet layer traffic ignores the specifics of link layer traffic.

Inernet Protocol (IP) covers the standards for addressing and packet format.

Routing

Routers break IPv4 packets (more later) into fragments.

When a router receives a packet, it has 2 options:

Send that packet to the attached local network.
Forward that packet to a different router.

Routers have address tables that identify all connected networks/devices.

IP Packets

IPv4 Packet format:

header source destination data

header 12B: Packet metadata

source and destination 4B each: IP Address

data: MTU is 65,535 Bytes

IPv6 uses 8 Byte addresses IPv6 allows for jumbograms, with an MTU of 4GB

IPv4 packet header

type size fragment info ttl protocol header checksum

type 2B: IPv4 / v6, length of header

size 2B: Total size of the packet

fragment info 4B: full payloads may be broken into multiple fragments. Each packet will count the number of fragments and its individual fragment number.

ttl (time-to-live) 1B: Maximum number of hops a packet can make before reaching its destination.

protocol 1B: TCP / UDP (this is transport layer information)

header checksum 2B: only a checksum of the header, not the full packet.

IP Addressess

Every device connected to the internet has an I.P. (Internet Protocol) address.

They might look like this: 149.89.150.100 (IPv4)

or this: 0:0:0:0:0:ffff:9559:9664 (IPv6)

IP addresses are only needed when connected.

For the most part, organizations own blocks of IP addresses, and give them to their users as needed.

Because IP addresses are released in blocks, internet traffic can be routed in an efficient manner.

IPv4 / IPv6 differences

Address space: 2^32 / 2^128

Packet format: In addition to address size change, IPv6 packet headers have less information. They do not include a checksum or fragment information.

MTU: IPv6 can allot for an MTU of 2^32 (these are called jumbograms)

IPv6 puts more work onto Link layer devices and individual hosts (computers)

IPv6 does not fragment packets at all, relying on other devices to potentially take advantage of jumbograms

IPv6 has no checksum, assuming Link layer devices and hosts will check for data integrity if needed.

[back to top of page]

2024-12-21

2024-12-21T12:00:00+00:00

Routing

Routers break IPv4 packets (more later) into fragments.

When a router receives a packet, it has 2 options:

Send that packet to the attached local network.
Forward that packet to a different router.

Routers have address tables that identify all connected networks/devices.

IP Packets

IPv4 Packet format:

header source destination data

header 12B: Packet metadata

source and destination 4B each: IP Address

data: MTU is 65,535 Bytes

IPv6 uses 8 Byte addresses IPv6 allows for jumbograms, with an MTU of 4GB

IPv4 packet header

type size fragment info ttl protocol header checksum

type 2B: IPv4 / v6, length of header

size 2B: Total size of the packet

fragment info 4B: full payloads may be broken into multiple fragments. Each packet will count the number of fragments and its individual fragment number.

ttl (time-to-live) 1B: Maximum number of hops a packet can make before reaching its destination.

protocol 1B: TCP / UDP (this is transport layer information)

header checksum 2B: only a checksum of the header, not the full packet.

IP Addressess

Every device connected to the internet has an I.P. (Internet Protocol) address.

They might look like this: 149.89.150.100 (IPv4)

or this: 0:0:0:0:0:ffff:9559:9664 (IPv6)

IP addresses are only needed when connected.

For the most part, organizations own blocks of IP addresses, and give them to their users as needed.

Because IP addresses are released in blocks, internet traffic can be routed in an efficient manner.

IPv4 / IPv6 differences

Address space: 2^32 / 2^128

Packet format: In addition to address size change, IPv6 packet headers have less information. They do not include a checksum or fragment information.

MTU: IPv6 can allot for an MTU of 2^32 (these are called jumbograms)

IPv6 puts more work onto Link layer devices and individual hosts (computers)

IPv6 does not fragment packets at all, relying on other devices to potentially take advantage of jumbograms

IPv6 has no checksum, assuming Link layer devices and hosts will check for data integrity if needed.

Transport Layer

Computer to computer connection over a network.

Unconcerned with the individual hops of internet layer traffic.

In a program, a socket is the data type associated with a transport layer connection.

Network ports are used at the transport layer that allow a computer to have multiple open network connections at the same time.

TCP and UDP are transport layer protocols.

Network Ports

A network port is a computer specific sub-address that allows computers (that have a single IP address) to have multiple open netowkr connections.

There are 65,536 ports.

Ports < 1024 are well known, reserved ports used for specific services.

For example, an ssh server will have an open socket on port 22 (this is similar to a WKP). When a child process is created to communicate with a client, that process uses a different port, leaving 22 open for the listening server.

Regulated by the Internet Assigned Numbers Authority (IANA)

Transmission Control Protocol (TCP)

Reliable connection, guarantees delivery of data.
Data is considered a continuous stream that arrives in the order it is sent (which may not be true in the lower layers)
Connections are established using a 3-way handshake

User Datagram Protocol (UDP)

Does not require an explicit connection, no gaurantee of data delivery (at the transport layer).
Data is sent as discrete datagrams with a set size (as opposed to a continuous stream).
Datagrams may be dropped, or received out of order.
UDP connections are faster because they do not need to be reassembled at the other end.
Error checking can still be handled at an upper level.

Data Encapsulation

As data crosses from an upper layer to a lower one, layer-specific metadata is added to help aid transmission.

Application —> Transport

UDP or TCP headers are added, including network port information

Transport —> Internet

Data (including Transport headers) is packaged into IP Packets.

Internet —> Link

Packets (including IP and Transport headers) are packaged into Ethernet Frames.

Data decapsulation When data crosses back up a layer, the packaging for the lower layer is removed.

[back to top of page]

2024-12-22

2024-12-22T12:00:00+00:00

2024-12-22

Systems: Networking IV: A New Hope

Transport Layer

Computer to computer connection over a network.

Unconcerned with the individual hops of internet layer traffic.

In a program, a socket is the data type associated with a transport layer connection.

Network ports are used at the transport layer that allow a computer to have multiple open network connections at the same time.

TCP and UDP are transport layer protocols.

Network Ports

A network port is a computer specific sub-address that allows computers (that have a single IP address) to have multiple open netowkr connections.

There are 65,536 ports.

Ports < 1024 are well known, reserved ports used for specific services.

For example, an ssh server will have an open socket on port 22 (this is similar to a WKP). When a child process is created to communicate with a client, that process uses a different port, leaving 22 open for the listening server.

Regulated by the Internet Assigned Numbers Authority (IANA)

Transmission Control Protocol (TCP)

Reliable connection, guarantees delivery of data.
Data is considered a continuous stream that arrives in the order it is sent (which may not be true in the lower layers)
Connections are established using a 3-way handshake

User Datagram Protocol (UDP)

Does not require an explicit connection, no gaurantee of data delivery (at the transport layer).
Data is sent as discrete datagrams with a set size (as opposed to a continuous stream).
Datagrams may be dropped, or received out of order.
UDP connections are faster because they do not need to be reassembled at the other end.
Error checking can still be handled at an upper level.

Application Layer

Once a socket (transport layer connection) has been established, the data transmitted between computers is at the application layer.

Think about the data/payload section of an IP packet, anything inside that is application layer inforation.

Data at this layer will be packaged depending on the program or protcol being used.

Common application layer protocols: http, ftp, ssh, dns...

Many application layer applications have registered ports at the transport layer.

Data Encapsulation

As data crosses from an upper layer to a lower one, layer-specific metadata is added to help aid transmission.

Application —> Transport

UDP or TCP headers are added, including network port information

Transport —> Internet

Data (including Transport headers) is packaged into IP Packets.

Internet —> Link

Packets (including IP and Transport headers) are packaged into Ethernet Frames.

[back to top of page]

2025-01-03

2025-01-03T12:00:00+00:00

2025-01-03

Final Projects!

Your final project calendar:

Jan 2: Choose 0-2 partners and a topic. If you have 2 partners, your project should be larger in scope.
Jan 3: Create a proposal with your project outline and how you intend to create it.
Jan 6: You will be given your repos. Start work on your project.
Jan 21: (Tues) all projects due. Final commits 8am including links to your presentation.

The deadline is a cutoff date: students will lose write access to their repository after the deadline .

Any group-issues must be communicated to me ASAP. If a group member gets sick, or lazy, you must tell me what is going on.

`PROPOSAL.md` - due Monday in class.(You must place it into a repo during class.)

This should contain:

A list of Group Members
A statement of the problem you are solving and/or a high level description of the project.
A description as to how the project will be used (describe the user interface).
A description of your technical design. This should include:

How you will be using the topics covered in class in the project.
How you are breaking down the project and who is responsible for which parts.
What algorithms and / data structures you will be using and how they will be used..

A timeline with expected completion dates of various milestones of the project.

Documentation Specifications

Documentation 30% of your grade.

PROPOSAL.md as described above.
README.md
It should have the following information:
- Project Name
- Team name
- Team Members
- Broad description of project
- Link to your 5-8 minute video demo
- (Optional) List of any required libraries. Include instructions on how to install the libraries
- Instructions on how to install/use your project
DEVLOG.md
- This will contain a running list of updates to the project.
- Each entry should be timestamped and attributed to the student(s) that contributed to the update, along with a brief description of what was done.
- You must update this daily for full credit.
- The updates are indended to be made when you remember what you did and why.
makefile with compile and run targets. The exact instruction to use should be in the README.md file.

Git commits, messages, anti-procrastination:

commits/anti-procrastination 30% of your grade.

You have classwork and homework EVERY DAY to work on this project, there are no exceptions unless you are sick.

You must make commits daily for your project. There should be multiple commits per work session. A single commit in class and a single commit at home is not sufficient.

You will be penalized if you procrastinate, do not do work regularly, or do not commit regularly with meaningful commit messages.

It is acceptable to occasionally make few commits when stuck on a difficult part of the code, or when fixing bugs. This should not be the normal workflow however.

Presentation

Presentation 10% of your grade.

You and your partner(s) will give a 5-8 minute video presentation of what your program does, and how it works.

You will not edit code, or show code.

You will discuss what works, and what "almost" works in the case of features that are slightly buggy.

Do not talk about what you could have done with more time.

Your presentation should sound rehearsed/planned.

The actual project

The codebase and project 30% of your grade.

This includes if you worked on a sufficiently technical project of appropriate scope.

Does the project work, can it be run on the lab computers.

Grade Summary

30% The codebase and project

30% Documentation

30% commits/anti-procrastination

10% Presentation

Project Notes

You can use any external libraries you-d like, for example, gtk for windowing, sdl for games, etc. but you must do the following:

Make sure to mention any libraries in the README where you describe compiling instructions.
Check with me to make sure we can install them.
Make sure your program will run in the cs lab computers.
Make sure they don't trivialize a systems programming aspect of the project. (You get no credit for features that come from the libraries)

[back to top of page]

2025-01-06

2025-01-06T12:00:00+00:00

2025-01-06

Final Project Repo:

Please add your proposal to the repo today.

Please use formatting in your md files. The DEVLOG in particular should be formatted so that you do not have merge conflicts with your partner.

Please create the README and replace the placeholder text. You should not have a readme that says "Group Name" in it.

https://classroom.github.com/a/Vh67aNdh

Today/Tonight

Design the functions/comments/headers of your project. Organization is extremely important.

Create the skeleton files for your project and organize things as much as you can.

All members should try to commit from EACH device of theirs, so they know that their commits show up the correct way. Please check the commits from the website to ensure ALL devices commit using YOUR name and link to your profile. You should see your avatar icon next to your commits as well.

Make sure you are each working on your own dedicated branch, and merging onto main when you have good code.

The only place you should commit changes directly to main would be the DEVLOG, and README files.

[back to top of page]

2025-01-09

2025-01-09T12:00:00+00:00

2025-01-09

select is a function that monitors multiple file descriptors, allowing a program to read from different sources.
select can also be used for writing to files, but we will not focus on that usage.
select will block on all the provided file descriptors, and return when any of them have data to be read.
select returns the number of file descriptors that have data to read, and then modifies a parameter to set which descriptors are available. If an error occured, return -1.
To use select, you create a set of potential file descriptors using the type fd_set. Most of what we want from select involves interacting with an fd_set variable by using various macros.
To use select we must:
- Create an fd_set variable.
  - fd_set desciptors;
- Clear the fd_set (FD_ZERO)
  - FD_ZERO( &desciptors );
- Add file descriptors to the fd_set (FD_SET).
  - FD_SET(listen_socket, &descriptors);
- Call select and wait until any of the provided descriptors are available.
  - select(max_descriptor+1, &desciptors, NULL, NULL, NULL);
  - The first argument is 1 more than the largest file desciptor in desciptors. This is an artifact of how select works.
  - The second argument is the fd_set of descriptors waiting to be read from.
  - The third and fourth arguments are for descriptors for other actions, like writing.
  - The final argument allows you to set a timeout, if NULL, select will block indefinitely.
  - Returns the number of file descriptors that are ready (this is usually 1).
  - Modifies desciptors to contain the desciptors that are available.
- Once select returns, loop through the potential file desciptors and determine which one is available (FD_ISSET).
  - FD_ISSET(listen_socket, &desciptors);
- If you are going to use select multiple times, you-ll have to repeatedly zero out the fd_set and add the descriptors to it, since select modifies it.
- It is usually a good idea to have a backup fd_set to keep all your descriptors in.

Putting it all together for a program that reads from a socket or stdin:

fd_set read_fds;
int listen_socket, client_socket;
char buffer[100];

FD_ZERO(&read_fds);
//assume this functuion correcly sets up a listening socket
listen_socket = server_setup();

//add listen_socket and stdin to the set
FD_SET(listen_socket, &read_fds);
//add stdin's file desciptor
FD_SET(STDIN_FILENO, &read_fds);

int i = select(listen_socket+1, &read_fds, NULL, NULL, NULL);

//if standard in, use fgets
if (FD_ISSET(STDIN_FILENO, &read_fds)) {
  fgets(buffer, sizeof(buffer), stdin);
}
//if socket, accept the connection
//assume this function works correctly
if (FD_ISSET(listen_socket, &read_fds)) {
  client_socket = server_connect(listen_socket);
}

Server with Select demo

This server is compatible with telnet for testing. To make a similar client to go with it, you would use select as well, but whatever you type gets sent over the socket instead of printed.

This includes some code snippets to prevent the open port from lingering.

To connect to the server use the command telnet 127.0.0.1 PORT e.g. telnet 127.0.0.1 9998

You can exit out of telnet by pressing Ctrl+] and then typing quit

The server will EITHER: 1. process STDIN or 2. accept a connection and process the message from the client.

If you want to make a server that listens to multiple connections, you would have to engineer a more complicated soluition using this tool.

#include &ltsys/socket.h>
#include &ltsys/types.h>
#include &ltsys/socket.h>
#include &ltnetdb.h>
#include &lttime.h>
#include &ltstdio.h>
#include &ltstring.h>
#include &lterrno.h>
#include &ltunistd.h>
#include &ltstdlib.h>

int main(){
    struct addrinfo * hints, * results;
    hints = calloc(1,sizeof(struct addrinfo));
    char* PORT = "9998";


    hints->ai_family = AF_INET;
    hints->ai_socktype = SOCK_STREAM; //TCP socket
    hints->ai_flags = AI_PASSIVE; //only needed on server
    getaddrinfo(NULL, PORT , hints, &results);  //NULL is localhost or 127.0.0.1

    //create socket
    int listen_socket = socket(results->ai_family, results->ai_socktype, results->ai_protocol);\

    //this code allows the port to be freed after program exit.  (otherwise wait a few minutes)
    int yes = 1;
    if ( setsockopt(listen_socket, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(yes)) == -1 ) {
        printf("sockopt  error\n");
        printf("%s\n",strerror(errno));
        exit(-1);
    }

    int err = bind(listen_socket, results->ai_addr, results->ai_addrlen);
    if(err == -1){
        printf("Error binding: %s",strerror(errno));
        exit(1);
    }
    listen(listen_socket, 3);//3 clients can wait to be processed
    printf("Listening on port %s\n",PORT);

    socklen_t sock_size;
    struct sockaddr_storage client_address;
    sock_size = sizeof(client_address);

    fd_set read_fds;

    char buff[1025]="";

    while(1){

        FD_ZERO(&read_fds);
        FD_SET(STDIN_FILENO, &read_fds);
        FD_SET(listen_socket,&read_fds);
        int i = select(listen_socket+1, &read_fds, NULL, NULL, NULL);

        //if standard in, use fgets
        if (FD_ISSET(STDIN_FILENO, &read_fds)) {
            fgets(buff, sizeof(buff), stdin);
            buff[strlen(buff)-1]=0;
            printf("Recieved from terminal: '%s'\n",buff);
        }

        // if socket
        if (FD_ISSET(listen_socket, &read_fds)) {
            //accept the connection
            int client_socket = accept(listen_socket,(struct sockaddr *)&client_address, &sock_size);
            printf("Connected, waiting for data.\n");

            //read the whole buff
            read(client_socket,buff, sizeof(buff));
            //trim the string
            buff[strlen(buff)-1]=0; //clear newline
            if(buff[strlen(buff)-1]==13){
                //clear windows line ending
                buff[strlen(buff)-1]=0;
            }

            printf("\nRecieved from client '%s'\n",buff);
            close(client_socket);
        }
    }



    free(hints);
    freeaddrinfo(results);
    return 0;
}

Select with many clients.

If you want to keep connections to multiple clients, you need to maintain a list of file descriptors for the connections.

When one of the clients disconnects, you need to remove the corresponding file desciptor from the list.

It is surprisingly easy to do this! When a client disconnects, the socket will show up as having data to read with select, but when you read from it, there will be 0 bytes read. At that point you should remove that fd from your list.

[back to top of page]

2025-01-17

2025-01-17T12:00:00+00:00

2025-01-17

This is a REQUIRED Final Exit Survey.

ALL STUDENTS must fill this out over the weekend. If you do not fill this out your project grade will be in jeapordy.

go back to the top of the page

Systems Level Programming

2024-09-05

2024-09-05

Logistics

https://konstantinnovation.github.io

Resources

Homework 01

1. Forms:

2. Working environment

Mac

Windows

On all machines:

2024-09-06

2024-09-06

Class overview

Autonomy in my class

prerequisite terminal skills:

Terminal Refresher tutorial:

Homework 02

2024-09-09

2024-09-09

C Quickstart

SSH Keys

Homework

2024-09-10

2024-09-10

printf

2024-09-11

2024-09-11

Piazza questions are really important today

Binary

Primitive Types

Operator Precedence

Experiment!

2024-09-12

2024-09-12

Variables

Functions

Loops and conditionals have the same syntax as java!

Lab01

2024-09-13

2024-09-13

Compiling and Linking

Preprocessor

A few basic preprocessor directives

Homework:

2024-09-16

2024-09-16

Makefiles

Actual Example:

main.c

foo.h

foo.c

makefile

Update Lab01

2024-09-17

2024-09-17

Memory

Bits & Bytes

Computer Memory 101

Interacting with Memory in your Programs

2024-09-17

2024-09-17

Memory Addresses

Get an address of a variable:

Store an address

Get the value at an address:

Modify an address:

Modify the contents of the address:

Lab02

Do the following in a C program.

It will get a little tricky, so do things one step at a time, compile, run and test frequently.

2024-09-18

2024-09-18

SSH

the -fu suffix:

Etymology

More printf

Bash Scripts

Make

More `printf`

Array Indexing and `[]` Notation

When you use `""` to make a string literal :