Spectrally Tailored ‘HyperPixel’ Filter Arrays for Efficient Imaging of Chemical Compositions

We present a method for designing and fabricating 'HyperPixels': pixel filter arrays with custom spectral transmission properties that enable efficient imaging of specific chromophores or fluorophores. Multispectral imaging typically targets particular spectral bands to uncover the spectral properties of tissue in combination with spatial resolution. Unmixing spectral properties can uncover the type and quantity of chromophores or fluorophores due to their unique spectral absorption or emission. Pixelated filter arrays atop imaging sensors are low-cost techniques used to achieve multispectral imaging. Typically, the filter pixels exhibit bandpass spectral behaviour, allowing only a fraction of the incident light to reach the sensor. As a result, narrowband filter pixels trade off high spectral resolution with optical power loss. A way to avoid this issue and improve the signal to noise ratio (SNR) for individual targets is to use a filter array where individual pixels are matched to a target chemical compound's reflectance or emission spectrum. Simulations show a > 5-fold improvement in SNR under realistic noise conditions. These matched optical filters can also reduce the complexity of software or hardware spectral unmixing algorithms, offering the potential for real-time imaging of target compounds. We present a method for tailoring spectral transmission of individual pixels by building HyperPixels comprising multiple Fabry-Perot resonator subpixels with varying bandpass properties (FWHM = approximately 50-60 nm, thicknesses 75-150 nm) that collectively have the desired transmission spectrum. We used a numerical optimization process to design filter arrays for simultaneous detection of methylene blue and indocyanine green, commonly used in cancer diagnostics by clinicians. We then fabricated filters for indocyanine green detection using grayscale lithography with pixel sizes down to 5 µm and individual subpixels down to 0.5 µm and characterized them for their spectral properties.


INTRODUCTION
Spectral imaging is a powerful technique to non-invasively determine the composition of a sample by detecting and unmixing the spectral signatures of the key components; the spectral properties of the components such as reflectance or emission that act as a spectral fingerprint. [1][2][3][4] The ability to determine the presence of key chromophores and fluorophores is valuable in detecting or diagnosing diseases like cancer. [5][6][7] Spectral imaging can also be useful in other industries such as agriculture and food processing to determine the quality and freshness of foods. 8,9 A fast, low-cost technique used in spectral imaging employs multispectral filter arrays (MSFAs) atop an imaging sensor. [10][11][12] MSFAs are a mosaic of filters where each pixel detects light in a particular wavelength range. This can then be interpolated using a demosaicing algorithm to form images of the spectral properties at the selected wavelengths. 1,10,13,14 These images can then be unmixed to reveal the underlying composition of the object imaged, which results in a complex and computationally intensive pipeline. 4,14 Much of the incoming light is discarded if it does not fall within the spectral bands of the filters, resulting in increased noise compared to monochrome or broadband imaging.
'HyperPixels' comprise multiple Fabry-Perot resonator 12 subpixels with varying bandpass properties that collectively have the desired transmission spectrum. Simulations show a 5-fold improvement in signal to noise ratio (SNR) under realistic noise conditions. Matched optical filters can also reduce the complexity of software or hardware spectral unmixing algorithms, enabling the real-time imaging of target compounds. This has proven successful using multivariate optical elements in spectroscopy, 15 and the development of filters that form pixel arrays will enable compact imaging. We present techniques to design, fabricate, and characterize these HyperPixels, targeted for imaging indocyanine green fluorophores as a test case. This approach is highly versatile and can be adapted to a wide range of applications, including biomedical imaging.

METHODS
The HyperPixel filters were designed using numerical optimization and then fabricated using metals deposited by thermal evaporation, and electron beam greyscale lithography. Subsequently, they were characterized spectrally and morphologically and then analyzed to ascertain the accuracy of the design and fabrication process.

Quantification of SNR advantage
We first quantified the SNR advantage offered by the HyperPixel design with its sub-pixel Fabry-Perot resonators. It is well known that matched filters, in which spectral bands are admitted in proportion to the SNR in those bands, are optimal in terms of SNR and have been implemented optically using multivariate optical elements for efficient spectroscopy. 15 In contrast to the multivariate optical element approach, which requires multiple height-controlled layers, our HyperPixel design requires only a single height-controlled layer, though this height varies spatially. Though simpler to design and fabricate, this approach is less efficient than multilayer designs because the Fabry-Perot sub-pixels each only admit a single narrow wavelength band and cover only a small area of the full pixel.
To evaluate this trade-off between simplicity and SNR, we developed a simulation of a 4-element pixel filter array designed to distinguish between oxyhemoglobin, deoxyhemoglobin, water and fat. Two sensor designs were compared: in the first, each of the 4 pixels was matched to one of the four spectra using the HyperPixel approach, and in the second each of the 4 pixels was a simple Fabry-Perot bandpass filter with center wavelength optimized for maximum SNR in spectral unmixing. The simulated spectral transmission profiles of both designs were normalized to account for the power efficiency constraint of the HyperPixel design and ensure a fair comparison. We generated test spectra as a random mixture of the four target substances spectra: where stest is the test spectrum; sHbO2, sHb, swater, sfat are the discretized reflection/emission spectra of oxyhemoglobin, hemoglobin, water and fat represented as column vectors; w1, w2, w3, w4 are randomly selected weights between 0 and 1 such that w1+w2+w3+w4=1, and nspec is a vector of white Gaussian spectral noise. We then applied a simple linear detection and spectral unmixing approach to estimate chemical composition: where s1, s2, s3, s4 are the spectral transmission functions of the pixel designs being tested, nsensor is a vector of random Gaussian noise arising on the sensor (assumed to be thermal noise), and M is the optimal spectral-unmixing matrix. For any given test input, stest, SNR is computed by first setting nspec and nsensor to 0 and computing the signal ysig. Next, noisy versions are computed by adding in appropriately scaled noise vectors for nspec and nsensor to get y. SNR is then computed as: where the L1 norm 16 is used because the values in y and ysig represent optical powers. In the hyper-pixel case, s1, s2, s3, s4 will be normalized versions of the spectra of target substances, while in the reference case, these will be bandpass filters whose central transmission wavelength is selected to optimize SNR defined above. By computing SNR for 1000 randomly generated composition weights and varying the noise levels, we can compare the simulated SNR performance between the two designs.

Design
An ideal HyperPixel filter array would perfectly match the spectra of the target substances. However, due to fabrication constraints, we can only use a finite number of Fabry-Perot sub-pixels to create a desired HyperPixel transmission spectrum. Here, we opt to use a 10x10 array of subpixels giving a total of 100 sub-pixels per HyperPixel. Each subpixel has a bandpass response whose center wavelength and spectral width is estimated from simulations in Lumerical FDTD optical simulation software (Ansys), using a layer structure of substrate: Ag:resist:Ag:MgF2 (Fig 1a). This layer structure was optimized to improve the transmission and long-term stability of the filters. 10 The two Ag layers represent the mirrors of the Fabry-Perot cavity and are 22 nm thick. For ease of fabrication, the metal and MgF2 layers were the same for all sub-pixels, but the dielectric layer was varied ( Figure 1b). This was done so that a greyscale lithography process could be used to fabricate the dielectric layers using a SU8-2000.5 resist, eliminating the need to coat and develop the wafer after each exposure.
Using the simulated estimates of central transmission wavelengths and bandpass filter width, the spectra of all 100 subpixels can be combined to generate an aggregate spectrum for a HyperPixel. This spectrum is compared against the target spectrum (selected from the matched filter approach above), and a mean-squared error loss function is computed. We then use a basin-hopping global optimization algorithm in Python to minimize this loss function by optimizing the heights of each of the 100 sub-pixels. By limiting the considered wavelength range from 450 nm to 900 nm, we can, to a first approximation, neglect the effect of higher-order resonances of the Fabry-Perot filters, thus simplifying the design process. Once the optimal heights are selected, the sub-pixels are then laid out in a snake-like pattern in descending height order to reduce large step-changes in heights between elements to allow more reliable fabrication (Fig 1c).
For our first fabricated design, we chose to target detection of indocyanine green in transmission mode, commonly used in cancer diagnostics by clinicians. [17][18][19] The spectra of the HyperPixels were designed to match the spectra of this fluorophore when back-illuminated by a white light source, and thus comprises a notch-spectrum (due to absorption) plus an emission spectrum (due to fluorescence) plus some residual white light. This results in band-stop or notch-filter spectral transmission profile, a useful test case as these are significantly different to the standard bandpass response of Fabry-Perot cavities. Collectively, when modelled, these individual filters result in spectra that match indocyanine green ( Figure 1d).
Following an initial fabrication trial, it was found that due to variations in fabrication (use of different metal encapsulation or adhesion layers), FWHM of fabricated subpixels could be as large as 175 nm. Therefore, a second design was therefore produced that would still exhibit some band-stop behavior under this worst-case scenario (Fig. 1e).

Fabrication
In order to maximize robustness to fabrication variations, design 2 ( Figure 1e) was fabricated for two HyperPixel sizes, 10µm and 5µm, to demonstrate compatibility with realistic pixel sizes in commercial CMOS image sensors. Here, a thin Ti layer is used as an encapsulation layer to prevent tarnishing of silver and is kept to < 1 nm thickness. For the fabrication of the filters, 22 mm high precision glass coverslips (Thorlabs) were used as the substrate. Before processing, these were cleaned in ethyl lactate, acetone, methanol, and IPA, then dehydrated on a hotplate at 150 o C for 3 minutes. They were then placed in a thermal evaporator where a 1 nm Ti adhesion layer followed by a 22 nm silver layer and a < 1 nm Ti cap to avoid tarnishing was deposited. The silver was evaporated at a high rate to improve the film quality.
As soon as possible after evaporation, a film of AR-7720 (Allresist GmBh) was spin-coated at 4000 rpm for 60 s. The resist was diluted using PGMEA (1-methoxy-2-propanol acetate), so an initial film thickness of 165 nm was produced.
The sample was then placed in an electron beam lithography tool, NanoBeam nB5 (NanoBeam Ltd.), where a series of pixels with sizes of 0.5-10 µm were exposed. To vary the height of each pixel, the dose was varied using an earlier dose test which gave the contrast as 0.55. The exposed sample then had a post-exposure bake on a hotplate at 110 o C for 60 s followed by a bake in an oven at 90 o C for 2 hours; this later step reduces the surface roughness of the film. The resist was developed in a TMAH (Tetramethylammonium hydroxide) based developer for 90 s and then rinsed with DI (deionized) water. This resulted in a resist pattern with varying heights. The final step was to place it back in a thermal evaporator and evaporate 22 nm of silver followed by a 1 nm Ti cap. For the latter work, the Ti cap will be replaced with an MgF2 layer again deposited by thermal evaporation. 20

Characterization
Two characterization processes were used to determine the spectral transmission of the filters and the topographical properties of the filter, respectively, which were then analyzed and compared to assess the suitability of the filter.
For spectral characterization, an Olympus BX51 microscope was used to image the HyperPixel filter arrays. A UVvisible spectrometer (Ocean Optics HR2000+) was used to collect the spectral transmission data, while a digital camera (Lumenera Infinity-2 2MP CCD) was used to image the surface. The spectra collected were normalized using OceanView (Ocean Optics) software by collecting the bright and dark state spectra. The spectra of each sub-pixel were collected by translating the microscope slide that the sample was placed on using a 100x objective and centering the collection area over the center of the subpixel (Figure 2 a); 20 spectra were collected and averaged to reduce noise. The overall transmission of the HyperPixel was collected in the same manner, but a 5x objective was used since this resulted in a more extensive collection size centered on the overall HyperPixel.
For topographical characterization, the morphology of the samples was characterized using Atomic Force Microscopy (AFM) in tapping mode. The Al-reflex coated Si cantilever probes were used in the AFM (Asylum Research MFP-3D) and the system was used in tapping mode to determine the surface's height. The scan properties (scan speed, voltage setpoint, and drive amplitude) were optimized based on feature morphology. The data was then edited using the opensource software Gwydion (version 2.55) to remove horizontal scars, remove noise using a median filter (10 px), levelling the data to make the facets point upwards, and the minimum height was set to 0 nm. The minimum size was set to zero to allow relative analysis since it was impossible to determine the absolute height due to the sample preparation.

Analysis
The data was then analyzed to determine how the fabrication process affected the filters spectral properties compared to the design. First, the optical spectra at each of the sub-pixels are analyzed. To estimate relevant parameters, a bi-modal Gaussian model was fitted to each to each spectrum. This comprised a sum of two Gaussian functions (representing 1 st and 2 nd order Fabry-Perot modes) and a DC offset. The parameters estimated were the central wavelengths, amplitudes and widths of the 1 st and 2 nd Gaussian functions and a DC offset. Using the simulated data from Lumerical, it was then possible to estimate the cavity heights based on the 1 st order wavelength position. These parameters were then plotted against one another and against designed values to quantify performance.
Next, the AFM measured heights of subpixels were compared against designed heights. This was achieved by coregistering the measured AFM image with an image of the designed structure. The measured heights at every AFM point can then be plotted against the corresponding designed heights. A line is fitted to this data to give a slope an intercept. The intercept is relative to the AFM reference, but an absolute height can be estimated by finding the optical spectral peaks of the associated sub-pixels, using the model of Figure 1b to determine insulator thicknesses and then finding the mean offset from AFM heights. Together, the slope and intercept give a robust indication of whether the exposure times for the grayscale lithography are correct.

Simulated SNR performance
We analyzed the relative SNR of the HyperPixel design compared to a conventional bandpass design using four target substances (HbO2, Hb, fat, water). We found that for increasing levels of spectral noise, e.g. caused by the presence of additional substances or fluctuations due to movement/fluid flow, there was a significant SNR improvement for the HyperPixel design. This is illustrated in Figure 2. For example, with a spectral noise standard deviation of 0.1 and a sensor noise standard deviation of 0.001 there is a 5.5x SNR improvement for the HyperPixel design and this increases to 9.3x for a sensor noise standard deviation of 0.0001. The SNR advantage improves with increasing spectral noise because the matched filter is better at removing this spectral noise by better blocking bands with lower expected SNR. However, at low values of spectral noise, i.e., very 'clean' spectral mixtures, optimized conventional bandpass filter arrays may offer an advantage as they select only the bands where the discriminatory abilities between spectra are maximized. Figure 2. Comparison of SNR performance of recovered chemical concentration estimates (as defined in Eq. 4) as a function of spectral noise standard deviation. We note that the HyperPixel strategy offers a ~10 fold improvement over conventional sensing filters when large spectral noise is present. This advantage is lost at low spectral noise as noise is SNR is better optimized by selecting only the bands of highest discriminative power.

Filter height comparison
After topographically ( Figure 3) and spectrally (Figure 4) characterizing the samples, the heights of the individual pixels of the 10 µm sample were extracted and compared with the target height intended to determine discrepancies in the fabrication process. The resulting center wavelength was then used to compare how each subpixel's bandwidth (FWHM), the amplitude of first and second-order peaks measured, and DC offset of the transmission varied, as shown in Figure 5.  The AFM height vs target height fit has a slope of 0.36 (r 2 =0.97) for the 10 µm pixel size and 0.46 (r 2 =0.96) for the 5 µm pixel size, indicating that the grayscale lithography step may be slightly miscalibrated (Figure 5a). The relative difference in slopes across the two pixel sizes, as well as the offset (~ 60 nm) indicates there are some discrepancies in the heights of the samples depending on the HyperPixel size. This is likely due to a need to calibrate the exposure times of the smaller pixel. This could be improved by producing a calibration target, validated using the AFM, and iteratively adjusting the exposure settings of the e-beam. 10 The height of each subpixel relative to the center wavelength of its spectra is relatively linear (slope=0.333, and intercept= -227) with an R 2 of 0.936. This slope is significantly less than the Lumerical model for an ideal Ag:resist:Ag layer structure (slope = 0.37) which may be due to the presence of additional metal layers used for encapsulation. However, the absolute error within the range of wavelengths considered (400 nm -700 nm) is still acceptable ( Figure  5b). The bandwidth measured using FWHM had a small slope (-0.11, r 2 = 0.101), so it can be approximated by a constant value of 102 nm over the range of wavelengths used here (Figure 3c). The Gaussian amplitude of the first order peak significant downward linear trend (slope=-0.0158, and intercept=55.410) with an R 2 of 0.0745 (Figure 5d), which suggests that this should be accounted for in future iterations of the design process. The Gaussian amplitude of the second-order peak also showed an approximately linear correlation (slope=0.227 and intercept=-142.

Filter spectral comparison
Next, the overall transmission spectral of the full HyperPixels in both size ranges was compared (Figure 6a). From this, a shift in the spectra of about 108 nm to the right for the 5 µm samples compared to the 10 µm samples is observed, further indicating a need to calibrate the insulator and overall filter heights.
Based on the estimated heights of the sub-pixels (from AFM and spectral measurements), the transmission spectra for the HyperPixel was simulated using the height-to-wavelength conversion of Figure 1b and compared to the measured spectra, being generally in good agreement (Figure 6 b,c). However, to obtain the 'notched' behavior observed experimentally the FWHM of the spectra of the individual pixels must be reduced to 50-60 nm. The bandwidth (FWHM) of the 10 µm design sub-pixels were measured at about 100 nm; this discrepancy is likely because, at the high magnifications (and hence high numerical apertures) required to isolate sub-pixels in the optical microscope, incoming light enters the filter at larger angles resulting in angular blurring and associated spectral broadening. 21 This is also likely why the measured overall spectra appear broadened compared to simulations, especially for the 5 µm HyperPixel which is expected to be more heavily impacted by this effect. Figure 6. (a) The overall transmission spectra of the two HyperPixel sizes, showing a similar shape but with the 5 µm sample shifted to the right compared to the 10 µm sample. (b,c) The spectra of the filters can be simulated using the heights of the filters for the 5 and 10 µm samples resulting in good agreement with the measured spectra.

DISCUSSION
We present a method of designing and fabricating HyperPixelsstructured pixels that have transmission spectra tailored to detect particular substances. This is achieved by creating arrays of 'sub-pixels' of metal-insulator-metal Fabry-Perot cavities of varying heights. Our simulations show that under conditions of high spectral noise or interference, this design can offer as much as a 10x fold improvement in SNR relatively to a conventional bandpass multispectral filter array. However, we also find that in conditions of very low spectral noise, a design with optimally selected narrow bands may perform better. This provides a useful approach to designing filter arrays for snapshot multispectral imaging and shows that different designs may be preferred for different use-cases. For example, if large amounts of other unknown substances are present (i.e., source of spectral noise), which is often the case when measuring biological samples, the HyperPixel approach may prove superior.
We next presented an optimization-based design process to create practical designs for tailoring HyperPixel responses. This is achieved by using a 10x10 array of subpixels and selecting heights to create an aggregate spectral transmission profile that matches the target. These are then laid out in height order to aid fabrication. As an example case, we present the design for a HyperPixel that is sensitive to transmission imaging of indocyanine green. This has a band-stop profile which is not achievable using conventional multispectral filter arrays that offer only bandpass responses and thus demonstrates the versatility of this technique.
Finally, we fabricate the design using metal-insulator-metal Fabry-Perot cavities, with the heights set using grayscale electron-beam lithography. Two HyperPixel sizes are fabricated: 5 µm and 10 µm, which are compatible with many existing CMOS image sensor pixel sizes. For both designs, we observe band-stop behaviors as desired, but our characterization results show that further calibration of the grayscale process is required to better control the position.
Future work could see multiple HyperPixel designs fabricated into a filter array to enable optimal detection of fluorophores and chromophores of interest. This could be integrated into clinical imaging systems for non-invasive or minimally invasive imaging of chemical concentrations with improved SNR. This could lead to more effective real-time imaging biomarkers of disease. Other HyperPixels could target fluorophores such as methylene blue or endogenous chromophores such as oxygenated and deoxygenated hemoglobin. Other industrial applications for the filters include tailoring them to detect the freshness of products or the presence of environmental indicators.