Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
Last revisionBoth sides next revision
raw [2020/03/29 17:09] – [GRAY8 vs RTP] yairraw [2020/10/10 12:17] – [GRAY8 vs RTP] yair
Line 4: Line 4:
  
 **solution**: \\ **solution**: \\
-map the GRAY8 **one plane** source into a quarter width **four plane** RGBA output image\\ +a image formats are **just a convention**, internally they are byte(octet) streams. we can re-package a datastream to another format, as long as we mind our strides.  
-we will then remap it to Y8 on the receiving side. \\ + 
-the trick is to use gstreamer built in forma spec and let it optimize the remapping. +view the image format as just a stream of octets. here is our stream in Y8 format 
 +       +--+--+--+--+ +--+--+--+--+ 
 +       |Y1|xx|xx|xx| |Y1|xx|xx|xx| ... 
 +       +--+--+--+--+ +--+--+--+--+ 
 +and again, data quartered and represented in 8 bit depth and with stride of 4 
 +       +--+--+--+--+ +--+--+--+--+ 
 +       |R0|G0|B0|A0| |R1|G1|B1|A1| ... 
 +       +--+--+--+--+ +--+--+--+--+ 
 + 
 +note the above represents two sequential pixels.  
 + 
 +the trick is to use gstreamer built in [[https://gstreamer.freedesktop.org/documentation/additional/design/mediatype-video-raw.html?gi-language=c#formats|format spec]] and let that handle the conventions. 
  
 the [[https://www.collabora.com/news-and-blog/blog/2016/02/16/a-programmers-view-on-digital-images-the-essentials/|hammer]] the [[https://www.collabora.com/news-and-blog/blog/2016/02/16/a-programmers-view-on-digital-images-the-essentials/|hammer]]
Line 24: Line 35:
 00000000 (0x7f369c00a8c0): eb eb eb eb eb eb eb eb eb eb eb eb eb eb eb eb  ................ 00000000 (0x7f369c00a8c0): eb eb eb eb eb eb eb eb eb eb eb eb eb eb eb eb  ................
 00000000 (0x556228700560): eb eb eb eb eb eb eb eb eb eb eb eb eb eb eb eb  ................ 00000000 (0x556228700560): eb eb eb eb eb eb eb eb eb eb eb eb eb eb eb eb  ................
-┌────────┬─────────────────────────┬─────────────────────────┬────────┬────────┐ 
-│00000000│ eb eb eb eb eb eb eb eb ┊ eb eb eb eb eb eb eb eb │××××××××┊××××××××│ 
-└────────┴─────────────────────────┴─────────────────────────┴────────┴────────┘ 
 </code> </code>
-note: why white pixel is not FF=(255)?\\ +**note**: why white pixel is not FF=(255)?\\ 
-gstreamer follows YUV colorimetry when generating a GRAY8 test image and YUV doesnt use the entire 0-255 space. [[https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/issues/738|issue]]+gstreamer uses YUV colorimetry when generating a GRAY8 test image \\ 
 +and YUV doesnt use the entire 0-255 space. [[https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/issues/738|issue]] 
 + 
 +**note2**: this example is running on linux/debian, \\ 
 +to adapt to windows using powershell replace ''\'' with ''`'' and \\ 
 +place quotes around  ''video/x-raw, format=GRAY8, width=4, height=4'' 
 + 
 + 
 +====FakeRawPay==== 
 +gray8 raw video transported as RTP payload via a "fake" rgba parsing step\\ 
 +conforming to [[https://tools.ietf.org/html/rfc4175|RFC4157]]\\ 
 +pay 
 +<code dot> 
 +gst-launch-1.0 -v \ 
 +  videotestsrc pattern=white is-live=1 \ 
 +  ! video/x-raw, format=GRAY8, width=4, height=4, framerate=30/1 \ 
 +  ! rawvideoparse width=1 height=4 format=rgba \ 
 +  ! rtpvrawpay \ 
 +  ! queue \ 
 +  ! udpsink host=localhost port=5000 
 +</code>  
 +depay and back to gray8 
 +<code dot> 
 +gst-launch-1.0 -v \ 
 +  udpsrc port=5000 \ 
 +  ! "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)RGBA, depth=(string)8, width=(string)1, height=(string)4, colorimetry=(string)SMPTE240M, payload=(int)96" \ 
 +   ! rtpvrawdepay \ 
 +   ! queue \ 
 +   ! rawvideoparse width=4 height=4 format=gray8 \ 
 +   ! videoconvert \ 
 +   ! fpsdisplaysink 
 +</code> 
 +   
 + 
 ====strides and offsets==== ====strides and offsets====
-rawvideoparse let you specify the strides and offset, can we get this file to be parsed with that set ? \\ + 
-**(Syntax is <stride,ustride,vstride>)**+there are caveats to this approach (yuv). get the hammer and study the [[https://gstreamer.freedesktop.org/documentation/additional/design/mediatype-video-raw.html?gi-language=c#formats|design document]].  
 + 
 + 
 +{{ :pasted:20200328-193549.png?700 }}
  
 <blockquote>**pstride** only exist for simple pixel formats. It represent the** distance <blockquote>**pstride** only exist for simple pixel formats. It represent the** distance
Line 78: Line 123:
 </code> </code>
  
-{{ :pasted:20200328-193549.png?700 }} 
  
 <hidden extended format view> <hidden extended format view>
Line 125: Line 169:
 </hidden> </hidden>
  
-See raw video format [[https://gstreamer.freedesktop.org/documentation/additional/design/mediatype-video-raw.html?gi-language=c#formats|design document]] +<hidden more digging>
- +
- +
-====FakeRawPay==== +
-gray8 raw video transported as RTP payload via a "fake" rgba parsing step\\ +
-conforming to [[https://tools.ietf.org/html/rfc4175|RFC4157]]\\ +
-pay +
-<code dot> +
-gst-launch-1.0 -v \ +
-  videotestsrc pattern=white is-live=1 \ +
-  ! video/x-raw, format=GRAY8, width=4, height=4, framerate=30/1 \ +
-  ! rawvideoparse width=1 height=4 format=rgba \ +
-  ! rtpvrawpay \ +
-  ! queue \ +
-  ! udpsink host=localhost port=5000 +
-</code>  +
-depay and back to gray8 +
-<code dot> +
-gst-launch-1.0 -v \ +
-  udpsrc port=5000 \ +
-  ! "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)RGBA, depth=(string)8, width=(string)1, height=(string)4, colorimetry=(string)SMPTE240M, payload=(int)96" \ +
-   ! rtpvrawdepay \ +
-   ! queue \ +
-   ! rawvideoparse width=4 height=4 format=gray8 \ +
-   ! videoconvert \ +
-   ! fpsdisplaysink +
-</code> +
-   +
- +
-====raw parsing==== +
-<code c> +
-gst-launch-1.0 videotestsrc pattern=white num-buffers=1 \ +
-! video/x-raw, format=GRAY8, width=4, height=4 \ +
-! identity dump=1 \ +
-! filesink location=white.gray8 -q +
- +
-gst-launch-1.0  multifilesrc location=white.gray8 num_buffers=1 \ +
-! rawvideoparse format=rgba width=1 height=4 \ +
-! identity dump=1 \ +
-! filesink location=white.rgba -q +
-</code> +
- +
- [[https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/issues/738|issue]] with GRAY8 and gstreamer +
- +
 <file cpp /gst-plugins-base/gst-libs/gst/video/video-format.c> <file cpp /gst-plugins-base/gst-libs/gst/video/video-format.c>
  
Line 204: Line 204:
 MAKE_GRAY_FORMAT (GRAY8, "raw video", DPTH8, PSTR1, PLANE0, OFFS0, SUB4, MAKE_GRAY_FORMAT (GRAY8, "raw video", DPTH8, PSTR1, PLANE0, OFFS0, SUB4,
       PACK_GRAY8),       PACK_GRAY8),
-</file> [[https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/gst-libs/gst/video/video-format.c|video-format.c]] +</file>  
- +[[https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/gst-libs/gst/video/video-format.c|video-format.c]] 
-[[https://github.com/torvalds/linux/blob/master/include/uapi/drm/drm_fourcc.h#L174|fourcc]] on linux+</hidden>
  
 +[[https://github.com/torvalds/linux/blob/master/include/uapi/drm/drm_fourcc.h#L174|fourcc]] on linux\\
 +[[https://docs.microsoft.com/en-us/windows/win32/medfound/image-stride?redirectedfrom=MSDN|MSDN on stride]]