You could try a custom format for the cells;
Note though that all positive values will be displayed as Yes. Also, as this is just formatting other formulas (if any), will still consider the cell value to be 0 or 1 since you are changing the display format, not the actual cell value.
The other way to do this is to add another row/column with an if statement and use that column as your y-axis.
If the value of A1 is other than 0 or 1 the above will return a null value. This method also has the advantage that if you want to use other formulas to work with yes/no values, it will work, whereas the cell formatting will not.