Alif Semiconductor’s Post

Over the last 4 years while working on our flagship AI camera I got the opportunity to work and evaluate difference SoCs , ranging from Nvidia Jetson to Ambarella CV25. These SoC are designed to accelerate AI workloads, but how do we measure their efficiency? This blog post explores the key metrics and methodologies used to evaluate the performance of SOC neural network chips.

Very interesting- responsible Ai is not just about the human factors, but the energy and cost part too! “if you could theoretically completely run an LLM in only 13 watts on an FPGA (without a GPU), that would be a 38-fold decrease in power usage” This is the kind of innovation we need… if it proves out! #ResponsibleAi #SecureAi #VerifiableAi #PrivateAi

Great listen! (12 min.) Starts with comments on Agilex 5's AI features and then Sandra gives a great summary on Altera's current state and future direction. #acceleratinginnovators

A new research paper on scalable #MatMul-free language models could signal a major shift in the #AI accelerator market. By eliminating the need for matrix multiplication, these architectures cut memory usage by more than 10x and boost training speed by 25.6%! The heavy dependence on MatMul in #LLMs can largely be attributed to #GPU optimization for these kinds of tasks. This technology processes billion-parameter scale models at 13 watts by exploiting lightweight operations that are beyond the capabilities of current GPUs. This shift could open the door to alternative solutions like FPGAs and custom AI chips, driving greater competition, innovation, and diversification in the AI hardware market. https://lnkd.in/gwAw29mu #Semiconductors #MachineLearning #FPGA #AIChips #AIHardware

🌟 Exciting news! Microsoft is collaborating with Intel®, bringing the first Neural Processing Unit (NPU) powered by #DirectML on Windows. Find out more about how NPUs are transforming the world of AI and driving innovation across industries. Learn about our upcoming release of DirectML support for Intel® Core Ultra processors with Intel® AI Boost, the new integrated NPU. Explore how DirectML with NPU support will open new possibilities for AI on Windows and stay tuned for more exciting announcements. Read the full article here: https://ift.tt/DYnSmWR #AIinnovation #WindowsAI #IntelPartner

🔎 Check out the Performance Comparison of MCU AI Vision Boards! 𝙂𝙧𝙤𝙫𝙚 𝙑𝙞𝙨𝙞𝙤𝙣 𝘼𝙄 𝙈𝙤𝙙𝙪𝙡𝙚 𝙫2 𝙫𝙨 𝙂𝙧𝙤𝙫𝙚 𝙑𝙞𝙨𝙞𝙤𝙣 𝘼𝙄 𝙈𝙤𝙙𝙪𝙡𝙚 𝙑1 𝙫𝙨 𝙀𝙎𝙋32-𝙎3-𝙀𝙔𝙀 𝘿𝙚𝙫𝙚𝙡𝙤𝙥𝙢𝙚𝙣𝙩 𝘽𝙤𝙖𝙧𝙙 𝙫𝙨 𝙎𝙚𝙚𝙚𝙙 𝙎𝙩𝙪𝙙𝙞𝙤 𝙓𝙄𝘼𝙊 𝙀𝙎𝙋32𝙎3 𝙎𝙚𝙣𝙨𝙚 𝙫𝙨 𝘼𝙧𝙙𝙪𝙞𝙣𝙤 𝙉𝙞𝙘𝙡𝙖 𝙑𝙞𝙨𝙞𝙤𝙣 Himax Technologies, Inc. has recently launched the WiseEye2 HX6538-A processor, which incorporates dual Arm Cortex-M55 processors and an ARM Ethos-U55 microNPU. It also integrates Helium vector and floating-point extensions to enhance the convolution operation of the neural network model. 💡 Some features worth highlighting about the processor include: 💧 Award-winning low power consumption 💧 Significant uplift in DSP and ML capabilities 💧 Designed for battery-powered endpoint AI applications We've just announced the Grove Vision AI Module V2, the new kid on the block with the HX6538-A. If you're wondering how this new processor performs, check out this article on Hackster.io for more insights! ➡ https://lnkd.in/gXeHrWRS 🛒 Order your Grove Vision AI Module v2: https://lnkd.in/gaSQhjqB #tinyml #machinelearning #visionai #aisensor #smartsensor #visionaionedge #computervision

A fascinating usage of AI to find essential minerals and oil and gas... Learn how to use #pytorch to go from seismic data to a subsurface model. Try it live on Intel Developer Cloud along with Benjamin Consolvo Kelli Belcher. Just want to watch and try out the code later? That's great too! #oneapi #iamintel #unsdg

The modern #AI revolution is definitely compute driven. The various neural networks use many different learning mechanisms, all of them fundamentally compute driven. The Convolution Neural Networks (#cnn) use convolution kernels that in my mental model I equate to spatial summarization. Since image data is spatial in nature, CNNs do really well. Next up is the #attention mechanism used in #Transformers, which is vector dot product (cosine distance) and measure the degree of similarity, and by extension interaction between those vectors (token embeddings or intermediate representations). This similarity mechanism is especially suited for language. Then we have the various non-linearities such as #ReLU etc. which mimic a logic gate with an activation potential. Finally we have the fully connected (or in #MoE designs, routed) feed forward layers that represent mapping layers that map inputs into arbitrary & learned output spaces useful for the particular task at hand. All of these are compute heavy and increasing the density (i.e. the number of parameters) of models tends to blow up computational cost. That is why the new #Blackwell #GPU from NVIDIA has been so well received. The insight in its design is that the power requirements of the optical networking make #scaleup a more power efficient design than #scaleout. As Jensen Huang said at #GTC24, performance/watt is important at scale current #lmm models are at. https://lnkd.in/gKztTEDd

Direct GPU and HBM supply becoming mainstream as IC distributors expand AI ventures Follow for daily updates on market insights and trends. #reboundgroup #reboundelectronics #thesource #marketinsights #industrynews #techtrends #AI #AIcomputing

An expertly written article that spans the spectrum of quantization primer to a practical assessment of real #quantization implications in #AI model #training and #inference. "Nvidia, for one, claims that it [quantization] is the single largest component of the 1000x improvement in single-chip TOPS in the past 10 years, adding up to 16x. For comparison, the improvement in process technology from 28nm to 5nm has only been 2.5x!" In the world of #semiconductors, process was king. Looks like there's a new king. 👑 https://lnkd.in/dvUmHzQn