diff options
| author | Hans Verkuil <hverkuil@xs4all.nl> | 2017-03-06 11:23:15 -0300 |
|---|---|---|
| committer | Mauro Carvalho Chehab <mchehab@s-opensource.com> | 2017-03-22 10:16:17 -0300 |
| commit | 5086924ad4f219ad6498b2cb217a98637b0b55b9 (patch) | |
| tree | b19234532ffdef9582dc9fbd33a4a53f1cd2e210 /drivers/media/platform/vivid/vivid-vid-out.c | |
| parent | 733e009c37b6caa219d7057838582d1259c4554f (diff) | |
[media] vivid: fix try_fmt behavior
vivid_try_fmt_vid_cap() called tpg_calc_line_width to calculate the sizeimage
value, but that tpg function uses the current format, not the proposed (tried)
format.
Rewrote this code to calculate this correctly.
The vivid_try_fmt_vid_out() code was completely wrong w.r.t. sizeimage, and
neither did it take the vdownsampling[] factors into account.
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Tested-by: Vincent Abriou <vincent.abriou@st.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Diffstat (limited to 'drivers/media/platform/vivid/vivid-vid-out.c')
| -rw-r--r-- | drivers/media/platform/vivid/vivid-vid-out.c | 22 |
1 files changed, 14 insertions, 8 deletions
diff --git a/drivers/media/platform/vivid/vivid-vid-out.c b/drivers/media/platform/vivid/vivid-vid-out.c index 1a3373060954..0b1b6218ede8 100644 --- a/drivers/media/platform/vivid/vivid-vid-out.c +++ b/drivers/media/platform/vivid/vivid-vid-out.c @@ -390,22 +390,28 @@ int vivid_try_fmt_vid_out(struct file *file, void *priv, /* This driver supports custom bytesperline values */ - /* Calculate the minimum supported bytesperline value */ - bytesperline = (mp->width * fmt->bit_depth[0]) >> 3; - /* Calculate the maximum supported bytesperline value */ - max_bpl = (MAX_ZOOM * MAX_WIDTH * fmt->bit_depth[0]) >> 3; mp->num_planes = fmt->buffers; - for (p = 0; p < mp->num_planes; p++) { + for (p = 0; p < fmt->buffers; p++) { + /* Calculate the minimum supported bytesperline value */ + bytesperline = (mp->width * fmt->bit_depth[p]) >> 3; + /* Calculate the maximum supported bytesperline value */ + max_bpl = (MAX_ZOOM * MAX_WIDTH * fmt->bit_depth[p]) >> 3; + if (pfmt[p].bytesperline > max_bpl) pfmt[p].bytesperline = max_bpl; if (pfmt[p].bytesperline < bytesperline) pfmt[p].bytesperline = bytesperline; - pfmt[p].sizeimage = pfmt[p].bytesperline * mp->height; + + pfmt[p].sizeimage = (pfmt[p].bytesperline * mp->height) / + fmt->vdownsampling[p]; + memset(pfmt[p].reserved, 0, sizeof(pfmt[p].reserved)); } for (p = fmt->buffers; p < fmt->planes; p++) - pfmt[0].sizeimage += (pfmt[0].bytesperline * fmt->bit_depth[p]) / - (fmt->bit_depth[0] * fmt->vdownsampling[p]); + pfmt[0].sizeimage += (pfmt[0].bytesperline * mp->height * + (fmt->bit_depth[p] / fmt->vdownsampling[p])) / + (fmt->bit_depth[0] / fmt->vdownsampling[0]); + mp->xfer_func = V4L2_XFER_FUNC_DEFAULT; mp->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; mp->quantization = V4L2_QUANTIZATION_DEFAULT; |