CPS843/8307: Introductory Computer Vision

Instructor: Mark Fiala
The course is held Wednesdays 12-3pm in VIC203

NEWS:

Keep checking here for news in CPS843.

Marks online. Here are the latest (rev11) assignmenet marks as of May 1: cps843_win11_rev_11_nonames.xls, and for grad students cs8307_win11_rev_3_nonames.xls. Contact TA/GA and cc myself if you have discrepancies.

My personal apologies for the giant mess the emailing to the TA/GA has been. It is likely a spam filter setting on his email address. Last year the same email arrangement was not as bad, with the same TA. I had problems with Blackboard, it was down when I was setting up the course, I will try to get it or some other online file deposit system working for next year should I teach it again.

Assignment deadline extended to Friday April 22, further extension from April 15 -this time I really mean final extension! you must email or demonstrate all questions by then!

Assignment deadline extended to Friday April 15 -final extension, you must email or demonstrate all questions by then!

All bonus questions must be demo'd to instructor. This Friday 2-5pm I will be around (office or NCART lab or ENG211).

exam marks online.

Updated Assignment 3b with .PLY format info for 3D stereo bonus question.

Updated Assignment 4 with typos and error in Q1d (497 instead of 239 matches for SIFT).

Two pass connectivity notes - added page in lecture notes with clearer algorithm (see lec1 notes).

Changed erroneous Assignment 1 due date from Friday Jan 21/2011 to Friday Jan 29/2011.

IMPORTANT: Please use OpenCV version 1.0 instead of 1.1pre1a! 1.1pre1a was recommended to me but I'm finding problems, I find 1.0 works fine on several computers.

Course Description
CPS843 is the undergraduate version of the course, CS8307 is the graduate student version. The lectures are in common for both courses, but those enrolled in CS8307 have different assigment requirements.

Computer Vision is the automated processing of images, it can range from bar code readers, to counting cars on the highway, to classifying and counting cells under the microscope, to measuring position for augmented reality (AR), to guiding robots.

Computer Vision is different from image processing, where the final output is usually an image to be viewed by a human. It can be thought of as a form of AI focused on images and video. Please read below for more discussion of computer vision.

Textbook
I've decided to use the OpenCV textbook from O'Reilly for about 1/3 of the undergraduate course, and more for the graduate students. Links from Chapters and Amazon (different book covers but appear the same).
Course Management Form
  • CPS843/CS8307 Winter 2011 CFM
    • Lecture Notes
  • Lecture 0 - Intro to, motivation of Computer Vision, simple C image programming, brief linear algebra review. Associated C files and PGM images lec0_files.zip
  • Lecture 1 -thresholding, binary morphology, RGB Images, colour space conversion. Associated code, demo programs, and PGM images lec1_files.zip
  • Lecture 2 -Image formation, perspective projection, linear algebra, projection matrix. A linear algebra primer courtesy Shu, Roth. Open GL demo motion example .
  • Lecture 3 - Projection matrices cont. Code and images from class lec3_examples.zip .
  • Lecture 4 - Recovering Projection matrices. Code and images from class lec4_examples.zip .
  • Lecture 5 - Finding correspondences - comparing patches. Code and images from class lec5_examples.zip . Matching feature points paper by Vincent and Laganiere.
  • Lecture 6 - Stereo, Harris/KLT corners and other interest point detectors, SIFT. Code and images from class lec6_examples.zip . Slides on corner detection courtesy Shu, Roth. Original paper Shi, J. and Tomasi, C. 1994. Good features to track. (CVPR'94 conference pg. 593–600).
  • Lecture 7 - Kernel convolutions, edge detection, RANSAC, KLT tracker. Notes on kernel convolutions and filtering comp4900_lect5_filter1.pdf, comp4900_lect6_filter2.pdf, comp4900_lect7_feature1.pdf, and comp4900_lect8_feature2.pdf courtesy Roth, Shu.
  • Lecture 8 - Using OpenCV. Associated files lec8_files.zip .
  • Lecture 9,10,11 - homography, 2D projective geometry, ideal image, cross product matrices, epipolargeometry, fundamental matrices. Associated files lec9_10_files.zip , http://danielwedge.com/fmatrix . Epipolar notes courtesy Roth, Shu: comp4900_epipolar.pdf,
    • OpenCV
  • We are using the 1.0 version of OpenCV, instead of the most recent version 2.0. Earlier I said to use the 1.1pre1 version, I have since changed this to 1.0 after finding some bugs. The main OpenCV download page is href="http://sourceforge.net/projects/opencvlibrary/files/"> sourceforge download page . Here are some notes on setting include, lib, and DLL paths to compile, link, and run your code. Here is a link on installing OpenCV , the instructions refer to 2.0 but have been tested for 1.0 (thanks to Robert D for this). Here is a link on downloading a Microsoft Visual C++ 2005 Redistributable Package (x86) that fixed problems for one student (thanks to Benjamin J for this). Here are instructions for installing OpenCV2.0 (translated from Chinese by David Tam).
    • Utility Programs
  • hexedit.exe Hex editor - useful for looking at and editing binary files
  • Paint Shop Pro -image viewer program that can read PGM, PPM files download from Corel or if that doesn't work, here is a cached older version
  • Alternate 'paint' program for viewing PGM, PPM's from getpaint.net . Installation instructions (not present on the website) are: unzip the dll file in the plugin archive, and copy it to the %program files%/paint.net/FileTypes folder. (Thanks to R. Mazzatto for finding this).
  • WinRAR -for creating archives to email in assignments wrar341.exe
  • VRML viewer - for 3D data vrmlview.exe
  • Windows corner matrix viewer corner_detection_gui.exe -shows the computations behind the first stages of the Shi&Tomasi, KLT, Harris corner detectors
  • Windows homography matrix viewer view_homog_matrix.zip -one window per image, see homography mapping a point from one image to a point in the other. NOTE: I just changed this, it was reading the images in the wrong order. Now the .CFG file should contain "img1.pgm 1to2.hm img2.pgm" where the homography maps a point in img1 to a point in img2.
  • Windows fundamental matrix viewer view_fund_matrix.zip -one window per image, see fundamental matrix mapping a point from one image to a line in another.
    • Assignments
    Assignments will typically be emailed to our course TA Shahin Talaei at stalaei@ryerson.ca .
  • Assignment 1 Greyscale PGM, RGB PPM image manipulation. Binary image processing. Due end of the day Friday Jan 29/2011 Associated files assignment_1_files.zip
  • Assignment 2 2D and 3D coordinate system transformations, projection matrices. Due end of the day Friday Feb 11/2011.
  • Assignment 3A , Assignment 3B Projection matrix recovery, SAD, SSD, CSM, NCC, VCN patch comparison. Stereo. -Updated Mar 15 with PLY format info for 3D bonus question.
  • Assignment 4 KLT, FAST, SUSAN, SIFT interest point detectore, Matching, handling RGB. Updated Mar 15/2011 -biggest change being 497 not 239 matches for SIFT. Necessary files: Feature Extractor and and assignment4_files.zip .
  • Assignment 5 Homography and Fundamental matrices. Files for CS8307 and bonus questions assignment5_files.zip . This was last updated Apr 29.
  • Bonus Assignment Rev 0 . Earn more bonus marks, each mark = 1% onto your final grade! This was last updated Feb 9.
    • Practice Exams
  • cps843_8307_practice_exam_1.pdf,
    Course Description
    I personally believe computer vision, and its applications to robotics and AR, will change the world in the coming decades. Computer vision has long been a field of unsatisfied promise, turning pixels into understanding turned out to be a lot more difficult than first envisaged. Most computer vision simply hasn't worked for many years. However, since 2000 this has started to change, breakthroughs in algorithms and approaches have started to produce systems that really do work - one example is the human face detector in your digital camera.

    The course work will consist of programming assignments - these will be in part programming from scratch in C, and using the popular OpenCV (Open Source Computer Vision).

    In CPS843 we'll go through some of the basics of image processing (without going too deep into signal processing theory), and will learn some of the basic concepts in computer vision that have been proven for years:

    Below is a list of topics we will cover:

  • Image formation and projection matrices
  • Pose determination
  • Comparing image patches (correlation)
  • Filtering, including edge detection
  • line and circle finding
  • corner detection, and tracking (Harris corners, KLT tracker)
  • stereo (multi-camera) vision

    • We will learn some math useful for computer vision, such as:

  • matrix fundamentals
  • projection matrices and homographies
  • fundamental matrices and the epipolar constraint
  • RANSAC fitting of noisy data

    • We will also focus at least 40% of the course on the latest developments since 2000 such as:

  • SIFT, SURF feature detectors
  • MSER, Harris-Affine detectors
  • Viola-Jones face detector

    • And we will gain some brief exposure to topics such as:

  • visual SLAM (Simultaneous Localization and Mapping)
  • Photosynth, CBIR (Content Based Image Retrieval = image search engines)

    • Those in this course may be interested in our Ryerson CS Robotics Club .