hack news

Inaugurate provide USB C digicam with C mount lens, MIPI Sensor, Lattice FPGA, USB 3.0

     This put up goes but one other share in the DIY digicam projects which were doing since quite some time. On this put up I will exhibiting you subsequent profitable implementation of creating C mount excessive lens mount  USB C digicam. This implementation will take into accout absolute modular boards having dedicated Sensor board which will likely be modified if needed. 

Venture Video

Hardware Intention

There are three boards, USB, FPGA and Sensor board. 

Sensor board has Sensor itself alongside with its energy and Oscillator, Board has  Excessive Density connector to be ready to connect with FPGA/Host board, Excessive Density connector furthermore provide energy to the board has I2C, 4Lane MIPI with clock , I2C and furthermore few other lend a hand watch over indicators. 

FPGA board has on this case Lattice Crosslink NX LIFCL-40 in 256caBGA equipment with required energy regulators, This board furthermore has 2x 16Mybte RAM for capabilities that want extra memory, FPGA config flash memory is furthermore on this board this board has two excessive density connector. Board scuttle I2C model from Host accurate threw. 

USB Board on this case has CYUSB3014 USB 3.0 Superspeed controller, alongside with required energy and Memory, As this board has 3.0 USB C connector, so there is USB 3.0 mux is furthermore there to give a enhance to connector reversal 

Schematic 

Currently I take into accout accurate form one digicam PCB, This schematic reveals IMX290 IMX327 IMX462 PCB , all these 3 sensors take into accout same resolution and furthermore same PCB footprint.

FPGA board

USB 3.0 board

All of the boards are 6 Layer, All of them are 27 x 27 mm , while mounting holes are 22 mm appart

Digicam sensor Board 

IMX477 Sensor PCB

FPGA PCB

USB 3.0 Board

Meeting 


Hardware and Digicam Lens C mount

So as to mount a C mount lens I designed a mount in Fusion 360. as miniature threads on FDM printer are little no longer easy to tackle  There are a quantity of already made aluminum CS mount to C mount adapters can be found on-line, Raspberry Pi digicam furthermore comes with one such ring, I sold these CS to C mount ring and I designed share spherical my PCB to match these CS mount adapter ring to give me metallic thread. 

Enclosure

FPGA Manufacture

MIPI

What is MIPI, that you just would be in a position to presumably also google it to acquire out however fundamentally it is a interface specification for Displays and Digicam sensor to a utility processor.

Image blow mark block draw of MIPI.  On one side there is utility processor and other side is the peripheral. When peripheral is Digicam and CSI apply. though mipi is closed specification which plan one must be member of MIPI consortium to impact uncover admission to to elephantine specification. And membership of the consortium comes with a large be aware designate for people. Happily elephantine specification is already in the market accurate form a accurate keyword internet search away. DCS, CCS, DSI, CSI and DPHY the total specification can be found with accurate form jiffy of internet search.

MIPI DPHY Signal

The image reveals i bought from google reveals model level for MIPI , HS pushed by differential driver swings -200mV to +200mV at offset of 200mv. while LP model is a 1.2V lvcmos 

there are two varied modes of transmission , HS mode and LP mode, HS mode is for hiya drag conceal recordsdata while LP mode is for Low energy transmission. 

Receiver must detect when transmitter has long previous into HS mode and exited HS mode.

Image blow reveals how transmitter enter HS modes. 

Stage 0 : LP-11 mumble in the proven image is LP mumble.

Stage 1 : To uncover into HS mode driver drives LPdp low for Tlpx(minimal 50ns) and preserve in LP-01 (HS driver is tristate in LP 01).

Stage 2: Driver drives LPdn low for Ths-prepare (minimal 95ns) preserve in LP-00 , Later somewere in the midst of this stage goal machine will instructed it be 100R termination register. 

Stage 3: Now Aim is in HS, driver activates HS driver initiate sending obligatory zeros .

Stage 4: Ship obligatory 0xB8 sync byte and then payload.

As outlined earlier CSI , describes Packet constructing. How precisely bytes are packed on to wire in varied lanes configuration.

Image blow reveals packet constructing. 

There are Two kinds of Packets

Fast Packet: 4 Bytes (Mounted Length)

Long Packet: 6 – 655541 Bytes (Variable Length)

MIPI Fast Packet Structure
MIPI Long Packet Structure

Endianness

Bytes are despatched lsbit first and bytes in the packet are despatched LSByte first. 

CCS

Well-known fact with CCS when evaluating with DCS , CCS describes uncover interface to be I2C while with DCS commands are space over same HS line because the guidelines itself. 

However in case of digicam as per MIPI spec CCS is applied over a extra I2C line. 

CSI Single Body

Single Body from digicam is mark in the image blow. 

Digicam send a Body initiate packet 

Then send embedded line recordsdata which tells receiver concerning the stream 

Then image recordsdata line by line. 



Check Video

What originate this digicam sensor varied to digicam modules

IMX219 digicam is bare bone digicam sensor. What build plan when i teach bare bone digicam sensor is , there no longer powerful image processing occurring on the digicam die it self. Digicam sensor is Sensor array with Bayer filter on it , 10 bit ADC , clock machine , MIPI output driver and I2C controllable machine lend a hand watch over.

What does this plan for us as digicam sensor implementer. As my final aim is to interface this digicam to USB3.0 UVC with RAW YUV.  This digicam does no longer output YUV, overlook about YUV this is no longer any longer going to even output RGB. Digicam output is absolute RAW 10-bit ADC conversion end result from the Bayer filtered sensor array.

So scuttle first uncover RGB output from bayer raw recordsdata, a Debayer or demosaic desire to be performed. As soon as demosaic is finished we are in a position to take into accout RGB ready to be remodeled to YUV. And one we take into accout YUV it’ll also furthermore be transmitted to USB to be displayed.

What subsequent this digicam is no longer any longer going to take into accout is any automated lend a hand watch over over publicity. due to the digicam does no longer take into accout any intelligence to grab how dark  or sparkling scene is.  Resolution to this enviornment what raspberry pi put into effect is , Raspberry Pi gradually on every frame change analog impact register over I2C to regulate impact in keeping with how sparkling and dark scene is.

This digicam does no longer take into accout any white steadiness lend a hand watch over as successfully so host must build accurate form while steadiness compensations. To uncover accurate form colours out of image.

FPGA module Block Design 

FPGA block draw is mark in the image blow. This draw exclaim how total machine is applied and what essentially the critical ingredients what this draw does no longer exclaim is lend a hand watch over indicators and other miscellaneous stuff.

Byte Aligner Received Uncooked unaligned bits from DDR RX module outputs Aligned bytes, Bytes on MIPI lane does no longer take into accout any outlined byte boundary so this modules Seems for at all times fixed first byte 0xB8 on wire, as soon as 0xB8 is found, byte boundary offset is space, space output legit to energetic and initiate outputting accurate form bytes stays reset when recordsdata lane are in MIPI LP mumble .

Lane Aligner Receives plenty of lane, byte aligned recordsdata from mipi rx byte aligner @mipi byte clock  outputs lane aligned recordsdata in a multi-lane mipi bus, recordsdata on varied lane can also merely appear at varied offset so this module will wait till of the all lanes take into accout legit output initiate outputting lane aligned recordsdata so byte x from the total lanes outputted at same timescale

MIPI CSI Packet Decoder On the total a packet Stripper, eliminates header and footer from packet Takes lane aligned recordsdata from lane aligner @ mipi byte clock appears to be like for particular packet form, on this case RAW10bit (0x2B) RAW12bit (0x2C) RAW14bit (0x2D). Module outputs Stripped bytes in precisely the vogue they were bought. This module furthermore win packet length and output_valid is energetic as prolonged as enter recordsdata is legit and acquired different of bytes is aloof within the boundaries of packet length.

MIPI CSI RAW Depacker  Receives Upto 4 lane raw mipi bytes from packet decoder, rearrange bytes to output upto 8 pixel upto 16bit every output is one clock cycle delayed, since the vogue,  output_valid_o stays energetic handiest while  chunk is outputted

Debayer / demosaic Takes upto 8x upto 16bit pixel from depacker module @mipi byte clock output upto 8x upto 32bit RGB for every pixel , output is delayed by 2 traces Put in force Traditional Debayer filter, As debayer want pixel insist neighboring pixel that can also very successfully be on subsequent or earlier conceal line, so enter recordsdata is written onto RAM, handiest 4 traces are stored in RAM at one time and handiest three of the readable at any give time , RAM to which recordsdata is written to can no longer be learn. First line is anticipated to BGBG , 2nd line GRGR On the total BGGR format  

RGB to YUV Coloration Dwelling Converter Received upto 8 pixel RGB from the Debayer filter output upto 8pixel yuv422  Calculation is finished in keeping with integer YUV formula from the YUV wiki page 

Output reformatter Takes upto 8pixel yuv enter from rgb2yuv module @ mipi byte clock outputs 32bit 2pixel yuv output @output_clk_i , This block has RAM to take into accout output clock self sustaining of mipi clock, Output clock_clock will take into accout to aloof be rapid enough to be ready to uncover entire line payment of recordsdata sooner than subsequent line begins,  This implementation of Output reformatter outputs recordsdata which which intended to send out of the machine to a 32bit receiver reckoning on requirement this will likely be desire to be adapted as per the receiver 

Debayer / demosaic  Need little more attention than other modules , IMX219 datasheet incorrectly mention output as to be either GBRG or RGGB. 

However after wasting a entire lot time it grew to change into out digicam output BGGR .  IMX219 Digicam handiest output BGGR as outlined by the IMX219 Driver in linux repo MEDIA_BUS_FMT_SBGGR10_1X10,  Digicam datasheet incrorrectly defines output as RGGB and GBRG. Recordsdata sheet is inaccurate on this case.

To study my debayer, Iwas the utilization of inbuilt digicam test patterns. One key ingredient about IMX219 is Bayer filter form does impact test pattern as successfully. It looks esteem in Check pattern mode it outputs RGGB in preference to BGGR. Check pattern will take into accout R and B channel inverted when image take into accout accurate coloration.

Update: I take into accout discussed this agonize with raspberry pi , It grew to change into out flipping image appears to be the resolution, as soon as image flipped bayer output it accurate form for both recordsdata from sensor and test pattern. due to the flipping image does no longer impact bayer uncover of the test pattern.

MIPI RAW Packet Layout

ISP Pipeline Specs

No virtual restriction on supported frame rate or resolution. Examined more than 4K with IMX477 4056×3040. Can build 8K with spherical 30FPS and even elevated than that as prolonged as FPGA is rapid enough for needed frame rate and FPGA/Board has enough memory to be ready to retailer minimal 4 Line payment of pixels. Output Clock is self sustaining of MIPI clock. Without concerns Transportable code to Xilinx or any other FPGA, No Seller particular ingredients has been worn excluding for the PHY itself which will likely be modified by other vendor’s DDR phy and Embedded Block RAM. Ideally suited Debayer/Demosaic and Output reformatter want Block RAM. Block ram can furthermore be modified vendor’s RAM. Auto detection of RAW pixel width supporting varied digicam sensors and sensor modes without FPGA reconfiguration.

Slouch

  • Supports MIPI bus clock 900Mbitsps Per lane with upto 4 Lanes, Entire 3.6Gbitsps Sensor bit stream, Has been Examined upto 900Mbitsps with 8x Gear.
  • Pixel Processing pipeline with 2,4 or 8 Pixel per clock can attain more than 110Mhz with Lattice Crosslink-NX LIFCL-40 Excessive Slouch, So fundamentally Can process upto 880 MegaPixels per 2nd. With this will attain Round 120FPS with 4K resolution and spherical 30 FPS with 8K. And even 3000 FPS with 640 x 480 as prolonged as Digicam and MIPI Wire enables. With Different Sooner FPGA drag will likely be more.
  • FPGA Oputput Pipeline is decoupled from MIPI clock, runs on output clock, It feeds into Cypress FX3 32bit GPIF can build Max 160Mhz. Cyress FX3’s specs limits max GPIF clock to 100Mhz.

Configurability

  • Selectable max RAW pixel width

        FPGA Manufacture is configurable with parameters to give a enhance to pixel depth from RAW10 to RAW14 or Virtually any bit depth even 16bit RAW when it becomes a MIPI Specs. Parameter specify most pixel width that is supported while module auto detect equipment form at runtime with RAW14 selected as max pixel width, RAW10, RAW12 and RAW14 will likely be robotically detected and processed

  • Selectable different of MIPI lanes: With accurate form definition of Parameter be aware different of lane is furthermore configurable between 2 or 4 MIPI lanes.
  • Selectable Pipeline Dimension: Pipeline is Configurable with a parameter to Direction of 2,4 or 8 Pixel. 2 Pixel Per Clock is handiest in the market with 2 Lane MIPI, while 8 Pixel Per Clock is handiest in the market with 4 Lanes.
  • Selectable MIPI Gear Ratio: Particular person can have interaction climate to operate MIPI/DDR Phy in 16x or 8x Gear ratio. Most DDR/MIPI Phy supports 8x Gear while few build give a enhance to 16x equipment.
  • Selectable MIPI continuous clock mode
    Particular person and take into accout interplay between MIPI clock lp essentially based thoroughly Body sync or Body initiate and frame discontinuance packt essentially based thoroughly frame sync. Some MIPI cameras build no longer give a enhance to going to LP mode while frame clean happen, With this choice user can allow Body Open up and Body discontinuance detection, to take into accout a frame sync.
  • Selectable ROM essentially based thoroughly Sample Generator
    For ISP debuging ROM essentially based thoroughly sample generator can also furthermore be activated. Two ROM traces are there take into accout both even and ordinary line to elephantine image test.

Block RAM and DDR PHY IPs desire to be manually regenerated if Gear, pixel width , lane or PPC is modified.

Assessments

4 Lane 12 bit IMX477

        4056×3040 20 FPS Plump Sensor

        2028×1520 70 FPS Plump Sensor Binned 2×2

        2028×1080 100 FPS

4 Lane 10 bit IMX477

        1332×990 200 FPS Binned 4×4

        640×480 400 FPS Binned 4×4

2 Lane 12 bit IMX477

        4056×3040 10 FPS Plump Sensor

        2028×1520 35 FPS Plump Sensor Binned 2×2

        2028×1080 50 FPS

2 Lane 10 bit IMX477

        1332×990 100 FPS Binned 4×4

        640×480 200 FPS Binned 4×4

2 Lane 10 bit IMX219

        3280×2464 7 FPS

        1280×720 30 FPS

        1280×720 60 FPS

        1920×1080 30 FPS

        640×480 30 FPS

        640×480 200 FPS

        640×128 600 FPS

        640×80 900 FPS

4 Lane 12 bit IMX290/IMX327/IMX462

        1280×720 120 FPS

        1920×1080 120 FPS

2 Lane 12 bit IMX290/IMX327/IMX462

        1280×720 60 FPS

        1920×1080 60 FPS

Scope dangle 

MIPI 2 Lane Mode, Decoded Recordsdata reveals both lanes of lane aligned recordsdata IMX219 Plump frame one in every of the road, on ch2 is byte clock

A initiate frame MIPI equipment (0x00), Use for Body Sync with cameras the put Clock does no longer scuttle into Los angeles some stage in frame clean

A Body Kill MIPI equipment (0x01) , Use for Body Sync with cameras the put Clock does no longer scuttle into Los angeles some stage in frame clean

Point to difference of Snappy vs tiresome slew rate on GPIF port , CH1 reveals GPIF port recordsdata line and ch2 is ~100Mhz clok

IMX477 Check pattern mode 0 were elephantine is space to elephantine 0x7FF and other colours are zero, however pixel the put there’s no longer a blue coloration recent reveals excessive bits, Additionally Even line the put there’s no longer a blue at all reveals excessive bits, This would possibly presumably take into accout to aloof be saved in mind when matching colours 

Check image 

IMX219 Traditional Check Plump Body Colors Uncorrected 

IMX219 Plump Body Check pattern 5
IMX477 Plump Body Check Sample 2

IMX477 Plump Body Check Sample 3


Cypress FX3 Firmware

Firmware implementation with FX3 became as soon as quite easy. I take into accout attach the total resolution and framerate in the USB descriptor , As described earlier this form of digicam sensors are quite bare bone the total take into accout sensor element, PLLs and ADC . So this digicam sensor does no longer take into accout any lend a hand watch over over publicity, White-steadiness and even brightness, I take into accout applied handbook lend a hand watch over over USB UVC lend a hand watch over channel. it that that you just would be in a position to presumably also factor in to entirely lend a hand watch over digicam publicity and brightness.

Few stuff that you just would be in a position to take be aware of, cypress fx3 clock frequency desire to be space in 400Mhz mode to permit elephantine 100Mhz 32bit GPIF DMA switch.

Yet any other ingredient is though Cypress CYUSB3014 has 512KB RAM however handiest 224 KB and further 32KB is in the market for DMA buffer.

Having gargantuan buffer chunk is basically crucial due to the on every DMA chunk CPU intervention is anticipated to insert UVC header. As this is excessive performance utility less on the total CPU intervention is needed is most keen. So I take into accout space DMA chunk / UVC person packet to 32KB

Scope dangle Image blow reveals Channel 13 is the person DMA packet dangle and on Channel 12 mark CPU DMA create interrupt.

These Two scope dangle mark difference between 16KB DMA vs 32KB DMA

16KB DMA Dimension, CH13 DMA packet , CH12 CPU interrupt

32KB DMA Dimension, CH13 DMA packet , CH12 CPU interrupt


PCB and Schematic Provide is supplied in the Github Repo

//https://github.com/circuitvalley/USB_C_Industrial_Camera_FPGA_USB3

Related Articles

9 Comments

  1. [url=https://toradoltab.monster/]toradol 2017[/url] [url=https://modafinil.gives/]modafinil pharmacy uk[/url] [url=https://lasix.foundation/]furosemide 25 mg[/url] [url=https://albendazole.fun/]albendazole medicine[/url] [url=https://albuterol.wiki/]buy albuterol no prescription[/url] [url=https://neurontinpill.com/]gabapentin 400 mg cost[/url] [url=https://canadianpharmacy.solutions/]online pharmacy without scripts[/url] [url=https://lexapro.foundation/]canadian pharmacy lexapro[/url]

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button