Syllabus

Overview¶

EE 445: Fundamentals of Optimization and Machine Learning is an introduction to optimization and machine learning models motivated by their application in areas including statistics, decision-making and control, and communication and signal processing. Topics include convex sets and functions, convex optimization problems and their properties, convex modeling, duality, linear and quadratic programming, with emphasis on usage in machine learning problems including regularized linear regression and classification.

Goals¶

The objective of this course is to provide ECE students with the foundational mathematical concepts and theory that underpins modern optimization and machine learning algorithms. In particular, the course will provide a background in mathematical reasoning, and convex problem modeling and solving. The course also seeks to help students develop a mathematical understanding of how convex optimization tools are used in the design, and analysis of machine learning algorithms and optimization problems used in various ECE application domains including data science, decision-making and control, communication, and signal processing.

Learning Objectives¶

• Model basic machine learning algorithms using the language of convex optimization.
• Solve machine learning problems using tools from optimization.
• Gain experience with Python tools for Machine Learning and Optimization.

Textbooks:¶

• Main:
  • Optimization Models. Calafiore and El Ghaoui
  • Introduction to Applied Linear Algebra: Vectors, Matrices and Least Squares. Boyd and Vandenberghe
• Supplementary:
  • Linear Algebra, G. Strang
  • Linear Algebra Done Right, Axler free videos that go with book

Modules:¶

• Module 1: Linear Algebra Preliminaries

  This module will provide a review of vector spaces, norms and distances, linear independence, range and nullspace.

• Module 2: Least Squares Regression in ML

  This module will introduce the least squares problem, and different approaches to solving it. Applications in Machine learning will be given.

• Module 3: Singular Value Decomposition with Applications in ML

  This module will introduce the singular value decomposition (SVD), principle component analysis (PCA), and principle component regression. These are important tools for both analysis and prediction/forecasting in ML.

• Module 4: Geometric & Kernel Methods in ML

  This module will introduce geometric and kernel methods in ML including spectral clustering and kernel regression.

• Module 5: Convexity

  Time permitting, we will conclude with a module on convexity and its role in optimization and ML.

Prerequisites:

• Calculus sequence: Math 224 or Math 324
• Linear Algebra: Math 208 or Math 308 or Math 136 or AMATH 352
• Python: EE 241 or EE 235 or CSE 163

Grade Breakdown¶

Your percentage grade in this course will be weighted using these categories:

Homework¶

There will be weekly homeworks. All homeworks will be graded via the “self-assessment” process described on the homework page of the course website.

Python Exercises (Projects)¶

There will be several Python projects that you will complete in the Friday Sections in groups of size 1-3. You are welcome to work on the projects alone, but encouraged to work in teams. The goal will be to have 1-2 graded Python notebooks per module.

Exams¶

There will be two exams in this class: midterm (April 29^th) and final (June 3^rd). Both will be take-home exams in which you have 24 hours to complete the exam. You must work by your self.

Collaboration¶

Learning these ideas is challenging. We encourage you to discuss course activities with your friends and classmates as you are working on them. Ask questions, answer questions, and share ideas liberally; we want a class that is open, welcoming, and collaborative, where we can help each other build the highest possible understanding of the course material. However, to encourage everyone to learn the material, we impose restrictions on the information you may share with your classmates (see below).

Whenever you receive help from a third-party source, please cite any help that you receive as a method comment in code or as a footnote in writeups. There is no penalty for working with too many classmates. When in doubt, you should always err on the side of giving credit.

Ultimately, the goal of enrolling in this course is for you to learn this material, so that you will be prepared for exams, for research, for job interviews, etc. Engaging in academic misconduct does not help you towards that goal. If you are in doubt about what might constitute cheating, send the course staff an email describing the situation and we will be happy to clarify it for you.

Course Tools¶

This quarter, we will use a number of different tools in EE445. Reach out to the course staff if you have questions about using any of them.

Zoom¶

We will have a single Zoom link for the course. This will be used for remote lecture sessions as well as for office hours.

Canvas¶

Like many other EE courses, we are mainly using Canvas as a gradebook and a place to find Zoom/Panapto recordings. Please refer to the course website for most course information, including assignment specs and due dates. Recordings will be linked from the course website to Canvas or Google Drive where they are hosted.

Discord¶

Discord will be used as a discussion board. More details below. It is optional however I will be active on there answering questions on homework and lecture content.

Gradescope¶

We will only use gradescope for uploading the exam. All other assignments will be uploaded to canvas.

Getting Help from Staff & Peers¶

Discussion Board¶

Discord both serve as online discussion forums. For most questions about the course or materials, they are the right place to ask: the course staff read them regularly, so you will get a quick answer. However, if you need to send the course staff a private message, we recommend canvas or direct email instead of Discord’s private messages since these are easily missed.

To meet with us, the best way is to visit our virtual Office Hours. Many of us are available at other times by appointment. In Office Hours, you can ask questions about the material, receive guidance on assignments, and work with peers and course staff in a small group setting.

Office Hours¶

Office Hours are scheduled times where you can meet with members of the course staff to discuss course concepts, get assistance with specific parts of the assignments, or discuss computer science and/or life outside of it.

Course Climate¶

Extenuating Circumstances: “Don’t Suffer in Silence”¶

We recognize that our students come from varied backgrounds and can have widely-varying circumstances. We also acknowledge that the incredibly unusual circumstances of this particular quarter may bring unique challenges. If you have any unforeseen circumstances that arise during the course, please do not hesitate to contact the course staff or the instructor to discuss your situation. The sooner we are made aware, the more easily we can provide accommodations.

Typically, extenuating circumstances include work-school balance, familial responsibilities, health concerns, or anything else beyond your control that may negatively impact your performance in the class. Additionally, while some amount of “productive struggle” is healthy for learning, you should ask the course staff for help if you have been stuck on an issue for a very long time.

Life happens! While our focus is providing an excellent educational environment, our course does not exist in a vacuum. Our ultimate goal as a course staff is to provide you with the ability to be successful, and we encourage you to work with us to make that happen.

Disabilities¶

Your experience in this class should not be affected by any disabilities that you may have. The Disability Resources for Students (DRS) office can help you establish accommodations with the course staff.

Religious Accommodations¶

Washington state law requires that UW develop a policy for accommodation of student absences or significant hardship due to reasons of faith or conscience, or for organized religious activities. The UW’s policy, including more information about how to request an accommodation, is available at Religious Accommodations Policy. Accommodations must be requested within the first two weeks of this course using the Religious Accommodations Request form.

Inclusion¶

Our code and our projects are made better by considering a variety of viewpoints. Your course staff is committed to the values outlined in the University’s inclusiveness statement, and you are expected to uphold a supportive and inclusive learning environment.

If, at any point, you are made to feel uncomfortable, disrespected, or excluded by a staff member or fellow student, please report the incident so that we may address the issue and maintain a supportive and inclusive learning environment. Should you feel uncomfortable bringing up an issue with a staff member directly, you may consider contacting the Office of the Ombud.