Perform various operations on images (UTexture2D) in Unreal Engine (such as read, save, copy, clipboard...).

The following code is based on version UE5.3.

Source code

More source code details can be found in the UE Marketplace for the plugin: AIChatPlus

Reading: UE implements reading local system images as UTexture2D.

General method

This method is feasible in both the Editor and GamePlay modes, supporting image file formats such as PNG, JPEG, BMP, ICO, EXR, ICNS, HDR, TIFF, DDS, and TGA, covering most common image types.

The code is also very concise:

#include <Engine/Texture2D.h>
#include <ImageUtils.h>

UTexture2D* LoadImage(const FString& InLoadPath)
{
    FImage ImageInfo;
    FImageUtils::LoadImage(*InLoadPath, ImageInfo);
    return FImageUtils::CreateTexture2DFromImage(ImageInfo);
}

The result returned is UTexture2D.

Editor-specific methods

This method can also support additional types of images: UDIM texture maps, IES files, PCX, PSD.

The implementation of the code will be a bit more complex.

#include <Engine/Texture2D.h>
#include <Misc/FileHelper.h>
#include <Misc/Paths.h>
#include <UObject/UObjectGlobals.h>

#if WITH_EDITOR
UTexture2D* LoadImage(const FString& InLoadPath)
{

    TArray64<uint8> Buffer;
    if (!FFileHelper::LoadFileToArray(Buffer, *InLoadPath))
    {
        return nullptr;
    }

    const FString TextureName;
    const FString Extension = FPaths::GetExtension(InLoadPath).ToLower();
    const uint8* BufferPtr = Buffer.GetData();

    auto TextureFact = NewObject<UTextureFactory>();
    UTexture2D* Ret = Cast<UTexture2D>(TextureFact->FactoryCreateBinary(
        UTexture2D::StaticClass(), GetTransientPackage(), *TextureName, RF_Transient,
        NULL, *Extension, BufferPtr, BufferPtr + Buffer.Num(), GWarn));

    return Ret;
}
#endif

The implementation utilizes the FactoryCreateBinary function of UTextureFactory, which is capable of reading the additional file types mentioned earlier.

Translation: Copy: UE realizes copying UTexture2D

Sometimes, it is necessary to copy a UTexture2D and then modify the copied image. To copy the image, you need to use the engine's built-in function FImageCore::CopyImage. You just need to set the parameters for both images correctly and call this interface.

UTexture2D* CopyTexture2D(UTexture2D* InTexture, UObject* Outer, FName Name, EObjectFlags Flags)
{
    // src texture info, src ImageView
    FTextureMipDataLockGuard InTextureGuard(InTexture);
    uint8* SrcMipData = InTextureGuard.Lock(LOCK_READ_ONLY);        // Texture->GetPlatformData()->Mips[0].BulkData.Lock(InLockFlag)
    const int32 InSizeX = InTexture->GetSizeX();
    const int32 InSizeY = InTexture->GetSizeY();
    const EPixelFormat InFormat = InTexture->GetPixelFormat();
    const FImageView SrcMipImage(
        SrcMipData, InSizeX, InSizeY, 1, GetRawImageFormat(InFormat), InTexture->GetGammaSpace());

    // create dst texture
    UTexture2D* NewTexture = NewObject<UTexture2D>(Outer, Name, Flags);
    NewTexture->SetPlatformData(new FTexturePlatformData());
    NewTexture->GetPlatformData()->SizeX = InSizeX;
    NewTexture->GetPlatformData()->SizeY = InSizeY;
    NewTexture->GetPlatformData()->SetNumSlices(1);
    NewTexture->GetPlatformData()->PixelFormat = InFormat;

    // Allocate first mipmap.
    int32 NumBlocksX = InSizeX / GPixelFormats[InFormat].BlockSizeX;
    int32 NumBlocksY = InSizeY / GPixelFormats[InFormat].BlockSizeY;
    FTexture2DMipMap* Mip = new FTexture2DMipMap();
    Mip->SizeX = InSizeX;
    Mip->SizeY = InSizeY;
    Mip->SizeX = 1;
    NewTexture->GetPlatformData()->Mips.Add(Mip);
    Mip->BulkData.Lock(LOCK_READ_WRITE);
    Mip->BulkData.Realloc((int64)NumBlocksX * NumBlocksY * GPixelFormats[InFormat].BlockBytes);
    Mip->BulkData.Unlock();

    // dst texture ImageView
    uint8* DstMipData = static_cast<uint8*>(NewTexture->GetPlatformData()->Mips[0].BulkData.Lock(LOCK_READ_WRITE));
    const FImageView DstMipImage(
        DstMipData, InSizeX, InSizeY, 1, GetRawImageFormat(InFormat), InTexture->GetGammaSpace());

    // run CopyImage
    FImageCore::CopyImage(SrcMipImage,DstMipImage);

#if WITH_EDITORONLY_DATA
    NewTexture->Source.Init(
        InSizeX, InSizeY, 1, 1,
        FImageCoreUtils::ConvertToTextureSourceFormat(GetRawImageFormat(InFormat)), DstMipData);
#endif

    // cleanup
    NewTexture->GetPlatformData()->Mips[0].BulkData.Unlock();
    NewTexture->UpdateResource();

    return NewTexture;
}

Save: UE implements saving UTexture2D to a file

The core is to use the engine function FImageUtils::SaveImageAutoFormat, which is relatively simple to implement, but attention needs to be paid to failure retry scenarios.

void SaveImage(UTexture2D* InImage, const FString& InSavePath)
{
    if (!InImage) return;
    FImage ImageInfo;
    if (FImageUtils::GetTexture2DSourceImage(InImage, ImageInfo))
    {
        FImageUtils::SaveImageAutoFormat(*InSavePath, ImageInfo);
    }
    else
    {
        // if prev save failed
        // use ConvertTextureToStandard to change InImage to Standard format, and try again
        // then revert InImage's origin format
        // this is what FTextureMipDataLockGuard does
        FTextureMipDataLockGuard InImageGuard(InImage);

        uint8* MipData = InImageGuard.Lock(LOCK_READ_ONLY);
        check( MipData != nullptr );

        const FImageView MipImage(
            MipData, InImage->GetSizeX(), InImage->GetSizeY(), 1,
            GetRawImageFormat(InImage->GetPixelFormat()), InImage->GetGammaSpace());

        FImageUtils::SaveImageAutoFormat(*InSavePath, MipImage);
    }
}

Save: UE realizes saving UTexture2D to Asset

Save the UTexture2D in memory to an Asset, and it can be viewed in the Content Browser.

The core function requires the CopyTexture2D implemented above. We need to first create a copy of a new image and then call UPackage::SavePackage to save the Package containing the image as an Asset.

void SaveTextureToAsset(UTexture2D* InTexture)
{
    if (!InTexture) return;

    // open save asset dialog, choose where/which to save
    FSaveAssetDialogConfig SaveAssetDialogConfig;

    SaveAssetDialogConfig.DefaultPath =  FEditorDirectories::Get().GetLastDirectory(ELastDirectory::NEW_ASSET);
    SaveAssetDialogConfig.AssetClassNames.Add(UTexture2D::StaticClass()->GetClassPathName());
    SaveAssetDialogConfig.ExistingAssetPolicy = ESaveAssetDialogExistingAssetPolicy::AllowButWarn;
    SaveAssetDialogConfig.DialogTitleOverride = FAIChatPlusEditor_Constants::FCText::SaveAsAsset;

    const FContentBrowserModule& ContentBrowserModule = FModuleManager::LoadModuleChecked<FContentBrowserModule>("ContentBrowser");
    const FString SaveObjectPath = ContentBrowserModule.Get().CreateModalSaveAssetDialog(SaveAssetDialogConfig);

    if (SaveObjectPath.IsEmpty()) return;

    // init save info
    const FString PackageName = FPackageName::ObjectPathToPackageName(SaveObjectPath);
    const FString PackageFileName = FPackageName::LongPackageNameToFilename(PackageName, FPackageName::GetAssetPackageExtension());
    const FString PackagePath = FPaths::GetPath(PackageFileName);
    const FString TextureName = FPaths::GetBaseFilename(PackageName);

    // create new UPackage to put the new texture in
    UPackage* const NewPackage = CreatePackage(*PackageName);
    NewPackage->FullyLoad();

    // copy texture
    UTexture2D* NewTexture = UAIChatPlus_Util::CopyTexture2D(
        InTexture, NewPackage, FName(TextureName), RF_Public | RF_Standalone | RF_Transactional);

    // Generate the thumbnail
    // if not doing so, the texture will not have thumbnail in content browser
    FObjectThumbnail NewThumbnail;
    ThumbnailTools::RenderThumbnail(
        NewTexture, NewTexture->GetSizeX(), NewTexture->GetSizeY(),
        ThumbnailTools::EThumbnailTextureFlushMode::NeverFlush, NULL,
        &NewThumbnail);
    ThumbnailTools::CacheThumbnail(NewTexture->GetFullName(), &NewThumbnail, NewPackage);

    // setting up new package and new texture
    NewPackage->MarkPackageDirty();
    FAssetRegistryModule::AssetCreated(NewTexture);
    FEditorDirectories::Get().SetLastDirectory(ELastDirectory::NEW_ASSET, FPaths::GetPath(PackageName));

    // save args
    FSavePackageArgs SaveArgs;
    SaveArgs.TopLevelFlags = RF_Public | RF_Standalone;
    SaveArgs.bForceByteSwapping = true;
    SaveArgs.bWarnOfLongFilename = true;

    // save it
    if (!UPackage::SavePackage(NewPackage, NewTexture, *PackageFileName, SaveArgs))
    {
        UE_LOG(AIChatPlusEditor, Error, TEXT("Failed to save Asset: [%s]\n"), *PackageFileName);
    }
}

Clipboard: UE achieves copying images (UTexture2D) to Windows clipboard.

Windows-related functions

We will use the following functions related to the Windows clipboard:

• OpenClipboardOpen the clipboard and obtain the clipboard's handler.
• EmptyClipboardClear the clipboard and assign ownership of the clipboard to the current window.
• SetClipboardDataSet the clipboard data; the image data is sent to the clipboard through this interface.
• CloseClipboardAfter setting up the data, close the clipboard.

Image format of the clipboard

Standard Clipboard FormatIt introduces available clipboard formats, among which CF_DIBV5 can be used to handle images.

The specific definition required by CF_DIBV5 BITMAPV5HEADER structureHere, we choose the following configuration.

BITMAPV5HEADER Header;
Header.bV5CSType        = LCS_sRGB;
Header.bV5Compression   = BI_BITFIELDS;

UTexture2D settings

We have selected the color space of the clipboard image as LCS_sRGB above, which is the sRGB color space. Therefore, UTexture2D also needs to be set to the corresponding format beforehand:

bool ConvertTextureToStandard(UTexture2D* InTexture)
{
    if (InTexture->CompressionSettings != TC_VectorDisplacementmap)
    {
        InTexture->CompressionSettings = TC_VectorDisplacementmap;
        IsChanged = true;
    }
    if (InTexture->SRGB != true)
    {
        InTexture->SRGB = true;
        IsChanged = true;
    }
    if (IsChanged)
    {
        InTexture->UpdateResource();
    }
}

ConvertTextureToStandard is responsible for converting UTexture2D into the standard format: TC_VectorDisplacementmap (RGBA8) and SRGB color space. Once the UTexture2D format is aligned with the image format of the Windows clipboard, we can copy the image data to the clipboard.

Specific code

void CopyTexture2DToClipboard(UTexture2D* InTexture)
{
    if (!InTexture) return;

    FTextureMipDataLockGuard InTextureGuard(InTexture);
    // get InTexture info
    uint8* SrcMipData = InTextureGuard.Lock(LOCK_READ_ONLY);
    const int32 InSizeX = InTexture->GetSizeX();
    const int32 InSizeY = InTexture->GetSizeY();
    const EPixelFormat InFormat = InTexture->GetPixelFormat();
    const FImageView SrcMipImage(
        SrcMipData, InSizeX, InSizeY, 1, GetRawImageFormat(InTexture), InTexture->GetGammaSpace());

    // set clipboard Texture info
    const EPixelFormat OutFormat = PF_B8G8R8A8;
    const int32 NumBlocksX = InSizeX / GPixelFormats[OutFormat].BlockSizeX;
    const int32 NumBlocksY = InSizeY / GPixelFormats[OutFormat].BlockSizeY;
    const int64 BufSize = static_cast<int64>(NumBlocksX) * NumBlocksY * GPixelFormats[InFormat].BlockBytes;

    // set header info
    BITMAPV5HEADER Header;
    Header.bV5Size          = sizeof(BITMAPV5HEADER);
    Header.bV5Width         = InSizeX;
    Header.bV5Height        = -InSizeY;
    Header.bV5Planes        = 1;
    Header.bV5BitCount      = 32;
    Header.bV5Compression   = BI_BITFIELDS;
    Header.bV5SizeImage     = BufSize;
    Header.bV5XPelsPerMeter = 0;
    Header.bV5YPelsPerMeter = 0;
    Header.bV5ClrUsed       = 0;
    Header.bV5ClrImportant  = 0;
    Header.bV5RedMask       = 0x00FF0000;
    Header.bV5GreenMask     = 0x0000FF00;
    Header.bV5BlueMask      = 0x000000FF;
    Header.bV5AlphaMask     = 0xFF000000;
    Header.bV5CSType        = LCS_sRGB;
    // Header.bV5Endpoints;    // ignored
    Header.bV5GammaRed      = 0;
    Header.bV5GammaGreen    = 0;
    Header.bV5GammaBlue     = 0;
    Header.bV5Intent        = 0;
    Header.bV5ProfileData   = 0;
    Header.bV5ProfileSize   = 0;
    Header.bV5Reserved      = 0;

    HGLOBAL WinBuf = GlobalAlloc(GMEM_MOVEABLE, sizeof(BITMAPV5HEADER) + BufSize);
    if (WinBuf == NULL)
        return;

    HWND WinHandler = GetActiveWindow();
    if (!OpenClipboard(WinHandler)) {
        GlobalFree(WinBuf);
        return;
    }
    verify(EmptyClipboard());

    // copy InTexture into BGRA8 sRGB Standard Texture
    FTexture2DMipMap* DstMip = new FTexture2DMipMap();
    DstMip->SizeX = InSizeX;
    DstMip->SizeY = InSizeY;
    DstMip->SizeZ = 1;
    DstMip->BulkData.Lock(LOCK_READ_WRITE);
    uint8* DstMipData = static_cast<uint8*>(DstMip->BulkData.Realloc(BufSize));
    const FImageView DstMipImage(
        DstMipData, InSizeX, InSizeY, 1, ERawImageFormat::BGRA8, EGammaSpace::sRGB);

    FImageCore::CopyImage(SrcMipImage,DstMipImage);
    DstMip->BulkData.Unlock();

    // copy Standard Texture data into Clipboard
    void * WinLockedBuf = GlobalLock(WinBuf);
    if (WinLockedBuf) {
        memcpy(WinLockedBuf, &Header, sizeof(BITMAPV5HEADER));
        memcpy((char*)WinLockedBuf + sizeof(BITMAPV5HEADER), DstMipData, BufSize);
    }
    GlobalUnlock(WinLockedBuf);

    if (!SetClipboardData(CF_DIBV5, WinBuf))
    {
        UE_LOG(AIChatPlus_Internal, Fatal, TEXT("SetClipboardData failed with error code %i"), (uint32)GetLastError() );
    }

    // finish, close clipboard
    verify(CloseClipboard());

    delete DstMip;
}

Conversion between UTexture2D and Base64

This can be relatively simple to implement, let's dive into the code.

#include <Misc/Base64.h>
#include <ImageUtils.h>

UTexture2D* B64ToImage(const FString& B64)
{
    TArray<uint8> Data;
    FBase64::Decode(B64, Data);
    return FImageUtils::ImportBufferAsTexture2D(Data);
}

FString ImageToB64(UTexture2D* InTexture, const int32 InQuality)
{
    FTextureMipDataLockGuard InTextureGuard(InTexture);

    uint8* MipData = InTextureGuard.Lock(LOCK_READ_ONLY);
    check(MipData != nullptr);

    const FImageView InImage(
        MipData, InTexture->GetSizeX(), InTexture->GetSizeY(), 1,
        GetRawImageFormat(InTexture->GetPixelFormat()), InTexture->GetGammaSpace());

    TArray64<uint8> Buffer;
    FString Ret;
    if (FImageUtils::CompressImage(Buffer, TEXT("png"), InImage, InQuality))
    {
        Ret = FBase64::Encode(Buffer.GetData(), Buffer.Num());
    }
    return Ret;
}

Original: https://wiki.disenone.site/en

This post is protected by CC BY-NC-SA 4.0 agreement, should be reproduced with attribution.

Visitors. Total Visits. Page Visits.

This post was translated using ChatGPT. Please provide feedbackPoint out any omissions.