Addressing a System.IO.IOException error

[GRoston]

Following is a very simple program I wrote, using Visual Studio 2012, that clearly shows the error:

Imports System.Drawing.Imaging
Imports System.IO

Public Class Form1
    Private Sub Form1_Load(sender As Object, e As EventArgs) Handles Me.Load
        Using bmp As New Bitmap(800, 200)
            Dim gr As Graphics = Graphics.FromImage(bmp)

            For n = 0 To 3
                Dim fileName As String = "D:\Users\Gerry\Desktop\TmpImages\SmileFace" + CStr(n) + ".jpg"
                Using tmpBmp As Bitmap = New Bitmap(Image.FromFile(fileName))
                    gr.DrawImage(tmpBmp, n * 200, 0)
                End Using
            Next
            gr.Dispose()

            Dim finalName As String = "D:\Users\Gerry\Desktop\Combined.jpg"
            bmp.Save(finalName, ImageFormat.Jpeg)
        End Using

        Directory.Delete("D:\Users\Gerry\Desktop\TmpImages", True)
        Application.Exit()
    End Sub
End Class

The problem, which happens both in debug and stand alone modes, is that the folder TmpImages does not get deleted. The following is the error text:

System.IO.IOException: The process cannot access the file 'SmileFace0.jpg' because it is being used by another process.
at System.IO.Directory.DeleteHelper(String fullPath, String userPath, Boolean recursive, Boolean throwOnTopLevelDirectoryNotFound, WIN32_FIND_DATA& data)
at System.IO.Directory.Delete(String fullPath, String userPath, Boolean recursive, Boolean checkHost)
at Microsoft.VisualBasic.FileIO.FileSystem.DeleteDirectoryInternal(String directory, DeleteDirectoryOption onDirectoryNotEmpty, UIOptionInternal showUI, RecycleOption recycle, UICancelOption onUserCancel)
at Microsoft.VisualBasic.MyServices.FileSystemProxy.DeleteDirectory(String directory, DeleteDirectoryOption onDirectoryNotEmpty)
at bmpSave.Form1.Form1_Load(Object sender, EventArgs e) in C:\Users\Gerry\Documents\Visual Studio 2012\Projects\Learning and Testing\bmpSave\bmpSave\Form1.vb:line 22
at System.EventHandler.Invoke(Object sender, EventArgs e)

Note that in the code, I tried both Directory.Delete and FileSystem.DeleteDirectory and got the same error both times. The error relates to one of the files in the folder TmpImages that is simply opened for reading. Please suggest that which I should do to resolve this issue. Thanks.

- - - Updated - - -

I added the statement Close after the statement gr.Dispose. The error is no longer thrown, but the directory is not deleted. I also tried replacing the Directory.Delete with FileSystem.DeleteDirectory, but still no directory deletion.